There’s a 1 in 100 chance of you being a tech reader and not knowing the recent, rather interesting reports that came up from Consumer Reports w.r.t. the battery life of new Macbook Pros.
Last month, Consumer Reports turned down Apple’s MacBook Pro models as the battery test results came out a bit abrupt and varied from as low as 3.75 hours to as unacceptably high as 19.5 hours (I mean Come onnnn !). All the time these results were coming, there was some valid scepticism around the fact, that either there was some serious software issue from Apple which needed repair, or it had something to do with the Consumer Reports’ benchmarking methodology.
Turns out, both need a bit of correction on their own front. To understand things better, we’ll have to start with how Consumer Reports tested these Macbooks.
To test these Macbooks, Consumer Reports turns off Safari’s local cache for their browsing test. Now do note, that this setting is turned on by default and saves your Macbook a lot of battery life by storing webpages on your hard-drive and loading heavy elements like images and scripts from the hard-drive instead of the web — a phenomena commonly called ‘caching’.
Consumer Reports was turning this off to make sure it loads all the websites it is browsing over and over again, with all the images and scripts thus consuming much more battery and reducing the life.
This is where both Apple and Consumer Reports come out as defaulters. On Apple’s front, such rather intricate setting change from the publication resulted in finding an “obscure and intermittent bug”, that had led to the inconsistent battery life results. This bug only affected the systems when the caching setting was turned off, which is usually not the case.
On Consumer Reports front, it is perhaps not the right way to test battery lives. The publication represents to be testing battery lives on consumer’s behalf. Now no normal user would even know of turning off such a setting, let alone thinking of actually doing that. Unless of course, one is explicitly asked to.
With “normal user settings” enabled, Apple said that Consumer Reports achieved the expected battery life.
Consumer Reports also took effort to explain why it turned off the cache. The publication reported that it turns off the cache settings on the default browser for all laptops and not just Macs. It does so, so that it can browse thousands of different websites and not just the same ten websites again and again.
Well, the explanation would have seemed valid to me if Consumer Reports were one of those numerous layman benchmarking folks who put laptops and iPhones and smartphones through all sorts of extreme — clearly non-consumetric — tests to prove a certain point. If you are running the tests to publish a report that reflects a consumer’s behaviour, then its better to test them in their usual environments than creating a special one.
While using web browsing for battery benchmarks does reflect real life usage as most people use their computers to browse the internet, play music, and write documents, turning a particular setting — and a usual one at that — off, doesn’t make sense.
Apple has made it a point that this problem does not arise again by fixing the Safari bug in the latest MacOs Sierra Beta 10.12.3, seeded to developers and public testers.
Meanwhile, Consumer Reports said that it will complete its retesting of MacBook Pro battery life and report back with its’ update, when finished.
