[H]ard|OCP has a new article up about "Benchmarking the Benchmarks". In this article they prove that there is a huge difference between running a Timedemo (Render this demo as fast as you can) and playing the exact same demo (I am controlling my character in real time). They made their own demo (because you can't play the Timedemo included in Crysis) while recording the FPS, then they ran the demo they made. Here are those results:
So the HD 3870 X2 saw an increase of 22% ((41.2/33.7) - 1) between playing the demo and running the demo. Using the same demo and settings, they tested a 8800 GTX. The GTX improved by only 14% ((45.3/39.6) - 1). 14% and 21% are not the same numbers. They are not equal. Think hard about what that means, I will give you some time...
If running Timedemos on two cards really did tell you exactly how well both of the cards played the game, then those percentages would be exactly the same. Card A would get 22% more FPS when running the Timedemo when compared to playing the Timedemo, and Card B would get 22% more FPS when running the Timedemo when compared to playing the Timedemo. That is not what is happening though. The percentages are not the same. 22 does not equal 14 which means:
The difference in performance between two cards running a Timedemo does not equal the difference in performance between the two cards playing that same demo (or game). Therefore, Timedemo benchmarks in reviews are worthless to people who play games and are only helpful to people that like to run Timedemos as fast as they can.
Before I get any hate mail about how I am a [H]ard|OCP fanboy, let me say this. I like reading [H]ard|OCP because their content is rooted in logic and common sense. I am a fan of logic and things that make sense, which makes me a fan of [H]ard|OCP. Show me another site that is as logical and "true" as [H]ard|OCP and I will read it.