For the longest time, mobile benchmark apps have been the tool to measure the performance of the smartphone. While the apps had a history of boasting these numbers, we still have not been treading lightly with these numbers. The brands keep boasting on its marketing copies these benchmarking numbers against the competitors. Now a new report has surfaced online, which could change the dynamics of how we look at the benchmarking. This report accuses MediaTek of offering mobile benchmark cheating as a service. Let’s talk about in detail below.
MediaTek’s Mobile Benchmark Cheating as a Service
Started when Andrei from AnandTech benched a European unit of the Reno 3 Pro running MediaTek Helio P95 processor. The same device is also available in India. The P95 features Cortex A75 cores and was easily beating a more capable Dimensity 1000L processor found on the Chinese variant of the Reno3 in PCMark.
Asked for an anonymized version of PCMark, which showed a performance delta of over 30 per cent in benchmarks, and up to 75 per cent in subsets like writing workloads. Upon further inspection, in the device’s firmware files, there was a power_whitelist_cfg.xml file, most commonly found in the /vendor/etc folders of the phones. Within the file, there were a lot of popular apps with performance tweaks applied to them, including a list of benchmarking tools.
A power management tweak dubbed “Sports mode” was present on all these apps.
Interestingly, there were corporate versions of some benchmarks too, including Kishonti’s GFXBench. It didn’t have the sports mode power hint configured in the listing, but obviously it’s altering the default DVFS (dynamic voltage and frequency scaling – used to variate voltage and thereby the power to a processor), thermal and scheduler settings when the app is being used.
Huge red flag right there – not only are benchmarks for general users skewed, but also those for reviewers. So, what is Sports mode then? It’s a tweak added to ensure that for some apps, the SoC will run at its maximum possible frequency for longer.
Why is MediaTek being targeted?
Well, because this performance tweak wasn’t just on OPPO phones but phones from other vendors (Xiaomi, Realme, etc) running a MediaTek SoC too. Same power_whitelist_cfg configuration file present in their vendor partition too.
Interestingly, this config file is present in devices as old as the Sony Xperia XA1 from 2016, a smartphone which was powered by MediaTek’s Helio P20 processor. What’s more, this isn’t being done by the vendors but it stems from MediaTek SoC’s BSP (board support package). What’s more suspicious is that off late, recent updates to OPPO phones have been removing this from their firmware.
Is MediaTek the only company to do so? Not per the brand, who in their blog statement claim that the competition also does this. That said, the 765G variant of the Reno3 did not seem to be cheating the benchmarks. This also goes on to show that if OPPO or any other vendor was to be held accountable, then their SD-devices would also have cheated. But, it’s just phones with MediaTek CPUs.
The above story includes major inputs from Prateek Pandey.