Today Futuremark are releasing, for free, a version of 3DMark which runs on Android (3.0 and above). This is their next step in allowing us to compare the latest mobile devices to each other with iOS support to follow soon. We have had some time with the new Android edition of 3DMark and below we have a selection of benchmark results from various devices as well as some information on the software itself.
What is 3DMark for Android?
Well, who can say it better than Futuremark themselves?
"The Android version of 3DMark includes everything you need to benchmark Android smartphones and tablets. Developed in cooperation with Broadcom, Imagination Technologies, Intel, NVIDIA, Qualcomm, and many other world-class companies, 3DMark is the ideal benchmark for over 1,000 modern Android devices.
What's more, 3DMark scores can be compared across platforms. You can use 3DMark to test and compare the performance of devices running Android, iOS, Windows and Windows RT, giving your readers a better understanding of how new devices compare with the alternatives on other platforms.
The Android version of 3DMark includes a new test. Ice Storm Extreme raises the rendering resolution from 720p to 1080p and uses higher quality textures and post-processing effects to create a more demanding load for the latest smartphones and tablets. Ice Storm Extreme will soon be available in the Windows version of 3DMark too, giving you another way to compare the performance of Android and Windows tablets directly.
There are no separate Advanced or Professional Editions on Android. You will be using the same version of the benchmark that will be available worldwide, for free, from the Google Play store."
3DMark for Android starts with a very simple screen:
From here we can immediately run the two tests included in the benchmark.
A tab at the top then allows us to view details about our device and we can enter the settings which on Android are essentially limited to turning the demo segment on/off.
We can also view a leader board of devices which show how each model has performed in the tests.
Speaking of tests, here they are...
ICE STORM and ICE STORM EXTREME:
On Windows, Ice Storm uses a DirectX 11 engine limited to Direct3D feature level 9 (9_3/9_1 depending on optional shadow filtering support). On Android and iOS, Ice Storm uses OpenGL ES 2.0 and content, settings and rendering resolution are the same on all platforms and scores can be compared across Windows, Windows RT, Android and iOS.
Ice Storm runs at 1280x720 with normal texture resolution and post-processing quality. Extreme ups the resolution to 1920x1080 and increases texture resolution and post-processing quality. Given that there are a number of resolutions around for phones and tablets after the rendered scene is created it is scaled to the native resolution of the display. This is commonly called "offsceen rendering" and Futuremark feel it is the best approach for ensuring that devices can be compared fairly.
Futuremark also note that no vendor specific OpenGL ES 2.0 extensions are used, that textures are compressed using ETC and textures that require alpha channel are loaded uncompressed.
In terms of specific engine features we have:
Traditional forward rendering using one pass per light.
Scene updating and visibility computations are multithreaded.
Draw calls are issued from a single thread.
Support for skinned and static geometries.
Surface lighting model is basic Blinn Phong.
Supported light types include unshadowed point light and optionally shadow mapped directional light as well as pre-computed environmental cube.
Support for transparent geometries and particle effects.
16-bit color formats are used internally in illumination buffers if supported by the hardware.
So what about performance?
Well here we have a selection of results from the last few generations of hardware for comparison. Our test devices all have 1GB of memory and are running their latest OS. (NOTE: These are Euro/UK versions of the devices):
Galaxy Note 10.1
Exynos 4412 Quad Core A9-1.4GHz / Mali-400MP
Exynos 4412 Quad Core A9-1.4GHz / Mali-400MP
HTC ONE X
Tegra 3 - Quad Core A9- 1.5GHz / GeForce
Tegra 3 - Quad Core A9 -1.2GHz / GeForce
Dual Core Krait 1.5GHz / Adreno 225
Sony Tablet S
Tegra 2 - Dual Core A9 - 1GHz / GeForce
Thanks to Vodafone for their continued device support:
So there are some expected scores in the above tables, for example the Note 10.1 and Galaxy S3 scoring near identical results. Then there are some interesting figures such as the Tegra 3 scoring better than Exynos on gaming, but not physics. Then of course there are surprises such as the dual core Xperia T outperforming the rest of the devices in the graphics test which is more of a factor in the overall result and places it higher in the leader board.
So what are our thoughts on 3DMark for Android overall? Well so far it seems the app is off to a decent start. There are still a few stability kinks to be worked out, for example a few crashes on our S3 and there are mistakes in the spec/hardware database... even some missing mainstream devices such as the Note 10.1 but these are to be expected given the vast number of devices and configurations out there. The fact is that for some time now Android (and iOS etc) has needed a range of useful benchmarking applications and over time this may well become one of the key metrics when judging hardware for purchase.