|DRIVER INVESTIGATION - 23 APRIL 2004|
Over the past week or so you’ll all have been reading the reviews of the Geforce 6800Ultra, Nvidia’s new flagship card. We have not published the results of our tests with the 6800U yet and may or may not do so before an upcoming comparison article however during the course of our testing there were a few IQ specific issues which really caused concern. In order to provide the industry with a fair overall view of the 6800 IQ and performance we have taken the decision to publish these findings now.
Before we begin it should be stated that these tests were performed using the Forceware 60.72 drivers, the version provided by Nvidia to all press reviewing their new card. These drivers are still beta however they have an interesting approval. The following text is taken from the Futuremark website:
“Update April 14th, 2004: Futuremark has reviewed the ForceWare 60.72 drivers for the launch of the GeForce 6800 Series graphics cards at NVIDIA's specific request. We found that the 60.72 drivers used with the GeForce 6800 Series fulfill the optimization guidelines for 3DMark03 Build 340. Please note that the ForceWare 60.72 drivers are approved only for the GeForce 6800 Series graphics cards, and any future approved WHQL drivers will replace the 60.72 non-WHQL drivers.”
To us this statement means that these drivers are free from optimisations and therefore the results obtained when using these drivers can be fairly compared to all other results on cards using approved drivers. Considering our test system scored 11800 at default driver settings/3dmark03 settings this seems a stunning achievement, unfortunately our tests do not back up the statement produced by Futuremark.
The test system was built from scratch,
a format of the hard drive was performed (NTFS) and then Windows XP
was installed. Following the completion of the install the N-force
drivers were installed. The only updates applied were SP1a and Direct
X 9.0b. Following a reboot the Detonator 60.72 drivers were installed.
Next the benchmarking tools were installed and finally the hard drive
was de-fragmented. For all tests the Nvidia/ATI drivers were set to
best image quality. This included manually setting trilinear in mipmaps
and manually disabling Trilinear optimisations.
3DMark03 – Game Test 1
By viewing our flash animation you will have the reference rastinizer images compared to the results obtained when using the 6800U/9800XT. The reference image is rendered in software mode by 3dmark using features from the DirectX SDK. We have chosen frame 1010 as it shows 2 sections of image which have IQ issues.
On the Nvidia screenshot there are 2 areas we would like you to concentrate on initially. These sections are:
1. Underneath the right wing of the
On both of the above
sections you will see that the RGB image shows major differences between
the Referance image and the 6800Ultra image. This RGB effect is the
card rendering different levels of mipmap (texture) levels. Effectively
the driver/card is rendering part of the frame in a texture which
differs from that which Futuremark requested. This change in mipmap
level can allow the 6800Ultra to use a texture which is more suited
to its architecture and therefore increase performance of the card.
You will see that the image produced by the Radeon does not suffer
these anomalies and therefore is using the textures/quality requested
3DMark03 – Game Test 2
In Game Test 2 we have used the same
procedure as Test 1. The frame used is frame 614. The area to concentrate
on when looking at the flash animation or the Photoshop compares is
the left tunnel wall. Again studying the mipmap level used in each
image shows a Radeon image almost identical to the reference/rasterized
image. Comparing the 6800Ultra image to either of the two shows huge
discrepancies. Mainly in the red and green mipmap levels which move
by a large portion. Again this texture change can allow the 6800Ultra
to process lower quality textures, or different volumes of each texture
from other approved drivers which are using the requested textures/quality.
3DMark03 – Game Test 3
3DMark03’s game test 3 comparison uses
frame 507 as a comparison. There are various differences between the
Geforce 6800 Ultra image and the Reference/Rasterizer image however
the easiest to spot is the window fram at the rear of the room. Again
the area of mipmap is significantly different to that displayed by
the reference image. This combined with the other texture changes
throughout the image is another indication of the 6800 Ultra processing
a different workload to other cards.
Download HIGH RESOLUTION LOSELESS BMPS of all IMAGES (Broadband) 6 Meg RAR
Download JPG LOSSY IMAGES 2.7 Meg RAR
Test 1 - Download Photoshop Layered Documents 1.6 meg RAR
Test 2 - Download Photoshop Layered Document 2 meg RAR
Test 3 - Download Photoshop Layered Document 3.2 meg RAR
Alternative live viewing
Real World Test – Max Payne 2
Max Payne 2 has been around for a while now, its by no means a dated engine however it should be a breeze for the 6800 Ultra to render, and should require no changes to texture/mipmap level in order to gain a performance advantage over other cards.
For the comparison we have taken a frame from the games first level. The section was chosen as it uses a relatively long corridor and so has a large level of detail extending into the distance and allows for mipmap changes over a longer distance than our 3Dmark tests.
The area which differs most noticeably
in Max Payne 2 is almost dead centre in the scene. As you look down
the corridor. The green square on the 6800 Ultra image are where the
card/drivers are changing the mipmap level. This change allows the
card to display the same scene as another card however with different
texturing. The reference image does not display this square, and neither
does the Radeon image. As you move down the corridor using the 6800
Ultra and the detail in the frame/distance becomes less the square
disappears and the normal/reference textures and maps are applied
to that section of the image.
Download HIGH RESOLUTION LOSELESS BMPS of all IMAGES (Broadband) 1.7 Meg RAR
Test 1 - Download Photoshop Layered Document 2.1 meg RAR
When I first read the reviews of the 6800 Ultra I was very impressed to see the problems which affected the NV3x being resolved, there were a few issues with IQ mentioned – such as those in Farcry however these were to be fixed in a future driver and appear to be legitimate bugs. I feel this is not the case however for the issues described above. 3Dmark as we all know is a synthetic benchmark, the purpose of which is to compare cards on a level playing field. Based on the changes in mipmap/textures on the NV40 this is not the case. It is disappointing that with such a technology lead over anything currently released that Nvidia have driver issues such as those described above.
Max Payne 2 raises a different question altogether. Its not synthetic and its really a user experience issue. When your playing away at 100+ fps I have to be honest and say that its not a noticeable change in IQ over the Radeon. You’d be hard pushed to say which image is optimised with the mipmap square. To me though, its not a matter of what I can see so much as the fact that this changing of textures is happening behind the users back. I have selected maximum quality on my £400 graphics card and I fully expect it to have Max Quality…not lesser quality than a much cheaper competitiors product.
The final issue is why have Futuremark stated that these drivers should be considered as approved? Its very clear to me that the 6800Ultra is not rendering the reference image as desired by 3Dmark, regardless of any performance impact and therefore isnt comparable to any other card. This in result is no different to the optimisations in previous drivers which Futuremark have frowned upon.
We have forwarded Nvidia and Futuremark links to this article and have requested their comments on the findings. Hopefully the answers to the issues raised will resolve our concerns.