Samsung has been found out: the Galaxy Note 3 recognizes benchmark programs and adjusts the clock speeds to show artificial performance.
A stupid contempt of consumers, but also a sign the mobile industry is now where the graphic cards makers where in the nineties. Bench*marketing* is now the weapon of choice and ATI (now AMD) and Nvidia used it when they found themselves at similar levels of performance.
Apparently the mobile cpu industry has now matured to a point where performance differences are too small and trickery is replacing honest tests. Let's hope this causes an outcry with consumers to stop this dishonest benchmarketing.
When ATI and Nvidia were found out, it caused a disruption of brand loyalty. Let's hope history repeats itself and consumers vote with their wallets to stop these unfair practices.
UPDATE: all Android players except Motorola are found out now http://www.anandtech.com/show/7384/state-of-cheating-in-android-benchmarks #Tech
We dig into the boosting shenanigans and find they apply specifically to benchmark apps.
I really can't see the problem with this. Not many people buy a phone because the processor is ten percent faster. Also Samsung is just scaling up the processor to run as fast as it can go so giving a true representation of the maximum performance of the chip.
+Edouard Tavinor If nothing were to be gained Samsung wouldn’t bother. Getting higher rankings in spec comparisons by manipulating the benchmarking process without public disclosure is deceitful in my book. It suggest a relation to to product performance instead of being just raw chip performance under laboratory conditions.
Another reason why Samsung fell out of my recommended list. The other being their new-fangled "region locking".
+Marc Jeuken how are they 'manipulating the benchmarking process'? That is the speed at which their phones run those benchmarks. It's absolutely honest.
+Edouard Tavinor They are manipulating the process by making the GPU run a special exeption mode only when a popular benchmarking program is run. This results in a speed difference of 20% when compared to running in normal mode. Other devices with the same processor don't pull this trick. That's how Ars noticed.
+Marc Jeuken other devices could. Samsung just thought of it first. All's fair in love and benchmarking.
I guess others have some decency left and Samsung lost that as well.
I don't agree that all is well in love and benchmarking +Edouard Tavinor It's deceit and what I call bench marketing
Optimizing for benchmarks is fair: the idea is that there is an underlying real world problem which profits from it. Just putting the CPU in a high power consumption mode if you recognize the specific name of the test is not.
didn't samsung do that already with the Galaxy S4 , where benchmark apps were recognized and internal CPU/GPU safe-guard restraints were temporarily turned off to gain more performance?
Yes +Otto Normalverbraucher they did that in the non US version of the S4.
Oh it's gamesmanship alright, but that's in the nature of the beast. I find it difficult to be anything other than amused by it. People know what chips it has and what version of Android it runs.
No +Edouard Tavinor nerds know. People only get the headlines.
+Max Huijgen people don't read benchmark results on fan sites. They may read tests in major newspapers, but usually not even that.
+Max Huijgen I think this is sensible. If the phone runs full throttle all the time, it will drain the battery. May be it might be a good idea to have a toggle for full power mode from the "Powever Saving" quicksetting option. I would not want my phone to be on bench mark mode all the time. Who knows what sorts of tweaks apple are doing.
+Able Lawrence the Note is only in benchmark mode when a popular benchmarking app is running. Under normal conditions not all four GPU cores are active. Which begs the question why it needs all those cores. Apple uses a dual core design and gets similar performance.
+Marc Jeuken The cores get activated by the operating system as soon as they are needed.
+Edouard Tavinor Yup, when benchmarks are run 😉
+Marc Jeuken Any multi-threaded application will result in the operating system considering using more cores, not just benchmarks. I'm not sure where you get that idea.
I did’t say they are only in use when benchmarks are run. But I do suspect they are rarely used because _’Samsung appears to be artificially boosting the US Note 3's benchmark scores with a special, high-power CPU mode’_ , and ‘benchmarking locks the CPU into 2.3GHz mode, the fastest speed possible, and none of the cores ever shut off.’ Under normal conditions most GPU’s are continually switched off, even when an unknown benchmarking app is run. Because ‘Benchmarks exist to measure the performance of a phone during normal usage, and a device should never treat a benchmark app differently than a normal app’ When the Samsung software detects a popular Benchmarking app they (not the code in the app) decide to open the nitro valve. Normally this is not done because it will either drain or fry the device.
+Marc Jeuken maybe Samsung has hacked the kernel to the extent of modifying the willingness of the kernel to apportion work to different cores, but I'd be surprised. In fact I'm not sure how much say the kernel has in this version of the arm architecture.
So in general, no. Switching off core throttling will have a minimal effect on benchmarking results, if any. Samsung is getting the other 20% through other means
+Marc Jeuken Note (X) comes into its own if you run videos and also do multi tasking at the same time. Or even play multiple videos which even Note 2 could
+Able Lawrence All screen sizes have their benefits. Playing multiple video’s concurrently is not something I’d do very often on a portable device that size though.
Turns out this was reported earlier. July 30 Anandtech concluded: ‘Samsung was only exposing its 533MHz GPU clock to certain benchmarks – all other apps/games were limited to 480MHz’
http://www.anandtech.com/show/7187/looking-at-cpugpu-benchmark-optimizations-galaxy-s-4
That's Sammy for ya
Nope +Zack Jenkins I should update this post, but I don't have the time, but all Android players except Motorola are found out now http://www.anandtech.com/show/7384/state-of-cheating-in-android-benchmarks
Oh dayyum! My life has been a lie… As I type this on a note 2 Lol. Looks like my next is a moto
But that moto failed the benchmarks +Zack Jenkins 😉
Well that's that I'm done using phones… I have a lg 8300 on my desk and I might just activate it.