HTC has openly admitted tweaking hardware performance to affect benchmarks, and now it's starting to pay the price. Futuremark, creators of the popular 3Dmark graphics and CPU benchmarking tool, has officially placed the HTC One M8 at the bottom of its list, below 1000 other devices.
Benchmark cheating became an issue last year when Samsung was discovered to be selectively boosting its hardware when benchmark app are run. The scandal led to the discovery of other manufacturers employing the same tactic, as well as the creation of anti-cheating tools and more stringent benchmarking rules. Though the topic has somewhat died down recently, and Samsung was reported to already be playing fair, it came up once again when the new HTC One M8 was announced.
Rather than denying the accusation as Samsung did, HTC openly admitted to the crime. But more surprising is the fact that HTC doesn't believe it's a crime at all. It defends its performance modifications by claiming that the HTC One M8 has, or will have, a High Performance Mode in the developer settings that squeezes out every last bit of juice from the device. It uses this mode when benchmark tests are run because, according to HTC, these tests are meant to determine the maximum performance of the device. The chart below shows how the HTC One M8 performs when running the stock 3DMark app from Google Play Store versus an equivalent but renamed APK.
Futuremark couldn't disagree more. The company behind one of the most popular graphics benchmark suites says that these tools are meant to gauge real-world performance and not maximum performance. The HTC One M8 violates Futuremark's rules that no device should detect a benchmark app and modify its performance based on that because it would otherwise be impossible to measure how a device will perform under normal circumstances. This latest incident may become a springboard for debate on whether benchmarks do or should test for real-world conditions or for theoretical capacity. So far, other benchmarking sites, like AnTuTu and GFXbench, have been silent on which point of view they agree with.
SOURCE: Futuremark