Zardon

Today we have seen several AMD Crossfire Preview articles published and after reading these today they raise several key points.

Firstly we have Pcperspective a very highly regarded Tech site who have been testing Quad Crossfire in their labs. What concerned me about this preview was this:

"I should start out my testing notes by being very up front: AMD had some requests for us when sending out the components for the CrossFireX testing system. First, they wanted the system to be showcased on an AMD platform: AMD Phenom processor and 790FX motherboard. Second, they had a handful of games that they wanted to show the performance and scaling of and asked that we show those titles for this particular article. Lucky for me, all of the games that AMD asked us to test (Bioshock, Call of Duty 4, Crysis and UT3) were already in my standard GPU testing suite, so that was a given. After that though, I was told we could do all the testing we wanted on any platforms, games, etc that we wanted.

The system that AMD provided is about what you would expect: AMD Phenom 9700 @ 2.6 GHz, MSI 790 FX motherboard with 2GB of Corsair DDR2-800 memory and a single Raptor 150GB hard drive with Windows Vista x64 installed. We used a beta driver revision that AMD provided and were off to the races."

Basically it looks like AMD have not only enforced several of the games to review, but have indeed supplied a full system for benchmarking their graphics cards. In the past we have been offered these systems ourselves but have always declined, stating that any components we test will be slotted into our own systems with our own installations. I am not saying that Pcperspective are commiting any foul play here, however did they install a fresh OS themselves negating any possible third party "tweaks" already installed on the system?

Sure this might sound like im being controversial, however I feel there are points to be made here, websites are really here to serve their public and results should really only be shown from comparible hardware and system installations that you, the public would have on your own systems. After all how many of you use a Phenom processor? Im guessing less than 5%.

Also today we have a "preview" from Hexus, they were invited to AMD's HQ in Toronto. They have had no control over the system build, the installation and state "To demonstrate just where ATI is in terms of CrossFireX four-GPU performance and how well it scales, HEXUS was invited to a pre-release showing under AMD-controlled conditions at its base in Toronto, Canada."

AMD controlled conditions? Is this how a reputable review site should be showing benchmark figures, again to potential customers, you guys?

"HEXUS can confirm that the numbers on the spreadsheet below, provided by ATI, do represent actual frame rates during testing, as we picked various benchmarks at random and they returned results within a minor standard deviation of what's shown below."

In fact the only performance figures again are from this spreadsheet (similar to the kind mailed to the press in PR emails), apparently gained from a Phenom powered system on a premade, preinstalled AMD system which resides in their HQ.

The article then ends with a picture of Greg Ellis, AMDs performance and benchmarking employee.

_____________________________

Driver Heaven will only publish a preview of AMD's Quad Crossfire (or in fact any other product) from our own system build in our own environment, because only by doing this can we be sure that we are presenting our readers with accurate and thorough analysis of a product. If it means we miss hits and take longer to do so, then so be it.

Alex

well said Z, agree 100%. quite disappointed with PCperspective tbh, they are a really good review site, but all that system testing with a phenom, just seems like kissing ass. Useless test results for 99% of the public (if not more).

Hexus? have been a joke for a long time, quite funny that 4 page PR email though.

geraldmango

yeah I agree. I mean both are pretty well known sites. the pcperspective one seems a little more balanced although I really dislike that a company can't just give the video cards to the site and say "ok review these". not "ok review these, but use this cpu and be sure to use these games". probably why they were one of the only sites to get the article up.

Hexus' 4 page thing is more like a factory tour article with them being given a pat on the back on the way out of the HQ with a PDF of benchmark figures to publish.

Looks reallyyyyyyy bad.

brutusmaximus

funny im just after reading the pcperspective article and it left a bit of a weird taste in my mouth. I mean hats off to AMD to be producing such a system, even if it looks extremely inefficient to my eyes, but the way the review was handled, not their best.

I never visit hexus so I had a look over there. that is absolutely dire, really, ethically terrible. To publish a spreadsheet of figures from AMDs testing and to present it as your own preview?

__________________

LukeK

Hexus are terrible, since Rys went to B3D their graphics cards articles are rank amateur level. That isn't a preview, its a PR email published on a website pretending to be a preview.

"ok guys, we will pay your flights to Canada, and then you publish what we tell you."

BlueMak

First of all, well done Zardon and DH, I completely agree with what you are saying.

IF I had a site (which I don't) that did hardware reviews I would do a preview if I could get my hands on the CPU, but, though it would be fair and all, I would go out of my way to not include the games "suggested" by AMD.
It's sad that AMD is moving in that direction, very very sad.

__________________
The people who are regarded as moral luminaries are those who forego ordinary pleasures themselves and find compensation in interfering with the pleasures of others(Bertrand Russell)"You go into Afghanistan, you got guys who slap women around for five years because they didn't wear a veil,You know, guys like that ain't got no manhood left anyway. So it's a hell of a lot of fun to shoot them." - Lt. Gen. James N. Mattis
This is slavery, not to speak one's thought. [Euripides-The Phoenician Women (c.411-409 B.C.)] http://www.macedonia.info/FALLACIESANDFACTS.htm Sic semper tyrannis.

humonous

Hexus did this before with an Intel preview. Basically they flew to Intel and posted benchmarks they got from hitting "run" on a benchmark script.

It really is highly unethicial and I emailed them at the time to tell them I wouldn't be back if this is how they were working. Clearly backhanders play a part in this. Glad to get a public announcement of Driver Heaven's stance on this. I would much rather wait for a proper review from you guys with proper testing conditions rather than you guys taking money and doing what a company tells you to do.

Alex

For sure! any site being forced to use a Phenom, which we all know are a last choice enthusiast CPU, deserve to take some flak. hexus didnt even have their own system, they probably hit a button to run a flyover or something then compared it to a spreadsheet AMD gave them (which they published verbatim!). What about if the system had a set of really beta drivers with IQ issues not apparent in the benchmark script?

Scandalous.

Most of these "tests" (and I say that in the lightest possible way), will be extremely CPU limited anyway. amongst other issues.

Vikingod

Well put Z, glad you made this point.

I honestly wouldnt have a problem with this, so long as the point is made in the article that it isn't "preview". If someone wants to make a trip to AMD to watch them demo a system and report on what AMD found, thats fine, but don't pass it off as anything but what it is. Likewise, if you want AMD to ship you a system and report on how it performed, thats fine, but you should be reporting on the system, not the individual components.

Bottom line, there should be full disclosure. Anything else isn't news, it's PR.

__________________

OldBuzzard

Z

I don't see the 'problem' with Pcperspective's review/test.

They clearly stated upfront what they were doing and why. As I read it, once they benched the 'AMD requested' applications, they were more than welcome to load up and test any other application that they wished.

Now, if they woud have HIDDEN the fact that the system was AMD supplied, and that they only did benches on applications that AMD 'allowed' them to...than THAT's a problem.

The produce reviewed is aimed at enthusiests, that are 'supposed' to be intelligent enough to know that all system components do play a role. Not just the GPUs.

I can see AMD's side of this. With the state of processor development right now, just how many reviews of Quad Crossfire are we going to see using an AMD processor? I'll bet that your review (like 99% of the rest of the sites) is going to be on an Intel system. Right? At least this way we get 'some' indication of how Quad Crossfire is working on an AMD system.

Also, look at what happened with the first 3870 x2 reviews. A lot of them were done with the early (wrong) drivers. And at least one 'highly regarded' site did so, without shame, even stating that they knew the new ones were available, but couldn't be arsed to re-test with the proper ones.

It wouldn't bother me one bit you see you use the same 'AMD supplied' system and run benches on it, as I'm sure you would make the same disclaimer, and make it perfectly clear what you did, and why you did it. No different than Pcperspective.

I do respect your side of this too, as I know that you don't wish to be seen as an <insert manufacturer name>whore, and that is as it should be.

Now do me, and the rest of the FLIGHT SIM LOVERS a favor, and when you DO do a Quad Crossfire review...load up FSX and let us see what it looks like in a real GPU/System killer

__________________
When looking for a reason as to why things go wrong, never rule out sheer STUPIDITY

_ə_

since they clearly stated what exactly was going on, I don't really see what the problem is. we know that it's not a real review, we know that they didn't use the most popular CPU, we know that it was on an AMD-controlled system. so we know enough to know this isn't the definitive word. unless you think we're too stupid to understand this isn't a real review, what's the big deal?

remember, this kind of stuff has happened in the past. tightly controlled systems showing some "preview" of a product. but it's clearly described as such and we take that into account when we look at the numbers. intel had a similar period with the conroe previews (when they showcased an intel and amd box controlled by them, running the benchmarks they wanted) note I said similar.


hehe didn't see your post old buzzard, you said it just as well

BlueMak

Then why isn't it stated that it is an advertisement? Why is it written in the form of a preview?

It reminds me what I see in some magazines and newspapers where there are pages that look like articles, but are nothing but glorified ads. They also incude text clarifying they are such. But, if they weren't trying to confuse the reader, then why follow this strategy and have it mimic a normal article (or preview in this case).

__________________
The people who are regarded as moral luminaries are those who forego ordinary pleasures themselves and find compensation in interfering with the pleasures of others(Bertrand Russell)"You go into Afghanistan, you got guys who slap women around for five years because they didn't wear a veil,You know, guys like that ain't got no manhood left anyway. So it's a hell of a lot of fun to shoot them." - Lt. Gen. James N. Mattis
This is slavery, not to speak one's thought. [Euripides-The Phoenician Women (c.411-409 B.C.)] http://www.macedonia.info/FALLACIESANDFACTS.htm Sic semper tyrannis.

Alex

Its an ethical problem, especially in the case of Hexus. You don't label something a **insert website name*"preview"..... then proceed to show a spreadsheet of benchmark figures run by the company who make the product. You think this is fine? I certainly don't and for all we know a few younger or less educated users reading this will think those figures are 100% accurate and base a purchase on it. If they published it as a tour or an interview, then fine, thats cool. but an offical site preview? Previews are classified as sites running their own benchmarks and tests on hardware in their own offices, the preview signifies pre release hardware, hence not a "review", a preview does NOT mean the company who make it hand out figures to sites and they publish them verbatim under their own site name.

Of course its not a real review, but its a "preview" under a site's name, its basically glorified PR or advertising. You might not be stupid enough to understand it, but are we to assume everyone who reads these is highly experienced and savvy? The pcperspective one as I said, isnt quite so bad, and I don't think Zardon was getting at them, just the way that AMD forced them to adhere to specific hardware sets. Which is more useful to the end user? running a series of tests on a hardware configuration that hasn't sold well (phenom) so they could get hits from an early article (and give AMD processor advertising) or to put in a core 2 as a reference point everyone can use to gain something useful from the results? Clearly you aren't quite as clever as you think if you think this is all acceptable. Pcperspective would never have used a phenom in the testing but they were TOLD to do so, they probably knew if they didn't say "yes", then the hardware would have went to another site until AMD found one who adhered to their rules. I again don't think this is fine.

so what if it happened in the past? quite a few things have happened in history that weren't particularly good or nice, does this mean we just keep accepting it 100 years into the future? the more we let companies interfere with reviews and presenting useful, accurate and ethically correct editorials to the public, the less these previews and reviews mean in the real world. Lets just be honest and call them 4 page advertisements.

Yes Hexus did that as well in the past and I called them up on it on their forums and emailed the guys in charge, I was basically told to F*&k off. What they are doing is NOT right. Im sure they love the handouts, but as you said, im not stupid enough to take it seriously, im also not stupid enough now to realise that anything they put up might be possibly tainted on some level.