UL Delists Huawei Phones For Cheating in Benchmark Scores
3DMark is a free Android benchmark app available from Google Play. You can benchmark your Android smartphone with 3DMark, then compare its performance with the latest models.
Earlier this week, AnandTech reported that several recent smartphone models from Huawei and its sub-brand, Honor, seem to be producing artificially high and misleading benchmark scores.
After testing the devices in UL's own lab, the company confirmed that Huawei's devices -- Huawei P20, Huawei P20 Pro, Huawei Nova 3 and Honor Play - breach UL's rules.
Delisted devices appear unranked, and without scores, at the bottom of 3D Mark's popular list of the best smartphones.
UL tested each model with the public version of 3DMark, available from Google Play, and a private version of 3DMark that is not available to the public or manufacturers:
UL found that the scores from the public 3DMark app were up to 47% higher than the scores from the private app, even though the tests are identical.
With the public 3DMark app, these devices appear to use a hidden "Performance Mode" that overrides the devices' usual power profile.
The difference in scores prove that the devices are recognizing the 3DMark app by name rather than adapting to the type of work in the test.
UL contacted Huawei with their findings, and they have pledged to implement a more transparent approach in a future update.
"Huawei is planning to provide users with access to "Performance Mode" so they can use the maximum power of their device when they need to," Huawei said.
Optional performance modes that can be set by the user are already available on some other manufacturers' models.
Tweaking a phone to optimize benchmark apps might produce some numbers that make some people drool, but those numbers don't correlate to the actual experience of using the phone. They might speak to how well the phone performs under heavy stress, like while gaming, but a better test is to just play a game with it and find out what happens.
0 Comments