Consumer Reports dropped its recommended buy rating for Microsoft’s Surface laptops saying 25% have problems within two years of purchase. But instead of putting on my schadenfreude pants and dancing to my Microsoft-toting friends misfortune, I’m questioning Consumer Reports’ reliability.
Consumer Reports is an independent organization that reviews washing machines, cars, toasters, computers, and many other products then makes buying recommendations. They said this is the first year enough data was available to rate reliability since Microsoft is a newcomer to the laptop and tablet market.
CR said in a statement, “New studies conducted by the Consumer Reports National Research Center estimate that 25 percent of Microsoft laptops and tablets will present their owners with problems by the end of the second year of ownership.”
The products that lost their recommended status are the Microsoft Surface Laptop (128GB and 256GB versions) and Microsoft Surface Book (128GB and 512GB versions).
The Apple to Surface Comparison
This is the part where Microsoft fans cry foul and Apple fans get smug and say, “You should’ve bought a Mac.” Except maybe we shouldn’t.
CR slammed the Touch Bar MacBook Pro after its release saying battery life was wildly inconsistent and in some cases lasted only a couple hours. Apple fans were up in arms, Microsoft fans gloated, and in the end CR’s testing proved to be problematic.
Instead of testing the laptop’s battery life in more realistic conditions, CR used the developer mode in the Safari web browser to change settings in a way that most users wouldn’t ever do. That triggered an esoteric bug causing erratic battery use, and ultimately CR’s poor rating.
Apple fixed the bug—something most users would never encounter—and CR used a different method for testing battery life. This time CR said the Touch Bar MacBook Pro’s battery consistently performed as expected. Getting to that point, however, required Apple’s help.
Which brings us back to CR pulling its recommended buy rating for the Microsoft Surface laptops. CR said,
A number of survey respondents said they experienced problems with their devices during startup. A few commented that their machines froze or shut down unexpectedly, and several others told CR that the touch screens weren’t responsive enough.
Consumer Reports Methodology
If I’m already questioning CR’s methodology for testing laptops and refused to take its original Touch Bar MacBook Pro rating at face value, it’s only reasonable I do the same for the change in the Surface laptop ratings.
Microsoft isn’t buying into CR’s findings, either. The company responded to the rating change saying, “Microsoft’s real-world return and support rates for past models differ significantly from Consumer Reports’ breakage predictability.”
It’s possible Microsoft is seeing a 25% failure rate on its laptops, just like it’s possible Apple uses batteries that fluctuate inconsistently between 3 and 19 hours of use time. Still, if I’m going to question CR’s methodology when testing something I like I better do the same for other products, too.
I think I’ll leave my schadenfreude pants in the closet for now.