Consumer Reports Just Slammed Microsoft’s Surface Laptops, but Maybe they Shouldn’t

2 minute read
| Editorial

Consumer Reports dropped its recommended buy rating for Microsoft’s Surface laptops saying 25% have problems within two years of purchase. But instead of putting on my schadenfreude pants and dancing to my Microsoft-toting friends misfortune, I’m questioning Consumer Reports’ reliability.

Microsoft Surface laptop loses Consumer Reports recommend rating

Consumer Reports pulls Microsoft Surface laptop recommend rating

Consumer Reports is an independent organization that reviews washing machines, cars, toasters, computers, and many other products then makes buying recommendations. They said this is the first year enough data was available to rate reliability since Microsoft is a newcomer to the laptop and tablet market.

CR said in a statement, “New studies conducted by the Consumer Reports National Research Center estimate that 25 percent of Microsoft laptops and tablets will present their owners with problems by the end of the second year of ownership.”

The products that lost their recommended status are the Microsoft Surface Laptop (128GB and 256GB versions) and Microsoft Surface Book (128GB and 512GB versions).

The Apple to Surface Comparison

This is the part where Microsoft fans cry foul and Apple fans get smug and say, “You should’ve bought a Mac.” Except maybe we shouldn’t.

CR slammed the Touch Bar MacBook Pro after its release saying battery life was wildly inconsistent and in some cases lasted only a couple hours. Apple fans were up in arms, Microsoft fans gloated, and in the end CR’s testing proved to be problematic.

Instead of testing the laptop’s battery life in more realistic conditions, CR used the developer mode in the Safari web browser to change settings in a way that most users wouldn’t ever do. That triggered an esoteric bug causing erratic battery use, and ultimately CR’s poor rating.

Apple fixed the bug—something most users would never encounter—and CR used a different method for testing battery life. This time CR said the Touch Bar MacBook Pro’s battery consistently performed as expected. Getting to that point, however, required Apple’s help.

Which brings us back to CR pulling its recommended buy rating for the Microsoft Surface laptops. CR said,

A number of survey respondents said they experienced problems with their devices during startup. A few commented that their machines froze or shut down unexpectedly, and several others told CR that the touch screens weren’t responsive enough.

Consumer Reports Methodology

If I’m already questioning CR’s methodology for testing laptops and refused to take its original Touch Bar MacBook Pro rating at face value, it’s only reasonable I do the same for the change in the Surface laptop ratings.

Microsoft isn’t buying into CR’s findings, either. The company responded to the rating change saying, “Microsoft’s real-world return and support rates for past models differ significantly from Consumer Reports’ breakage predictability.”

It’s possible Microsoft is seeing a 25% failure rate on its laptops, just like it’s possible Apple uses batteries that fluctuate inconsistently between 3 and 19 hours of use time. Still, if I’m going to question CR’s methodology when testing something I like I better do the same for other products, too.

I think I’ll leave my schadenfreude pants in the closet for now.

9 Comments Add a comment

  1. Jamie

    Their recommendations have definitely been spurious for awhile, now. Though to be fair, Windows laptops require about a kajillion times more attention and maintenance to get there than Apple’s products, I have seen a great many last for a very long time. I don’t see why the Surface line should be any different.

  2. Rick Allen

    The whole thing is just another click-bait headline. CR has lost all relevance in the Laptop testing that they do. I have not seen any other organization or written material that has led me to believe that Surface products are failure prone. Its clearly a shameless attention grab in the hopes that someone might still think they have anything to say

  3. wab95

    Consumer Reports is an independent organization that reviews washing machines, cars, toasters, computers, and many other products then makes buying recommendations.

    But not toaster fridges? Perhaps CR are just out of their depth here.

    Jest aside, this is the kind of balanced analysis and perspective one comes to expect here at TMO, an asset that should never be taken for granted, and a key reason I continue to enjoy this site.

  4. boltar

    There is a difference between initial quality/performance assessment and long-term reliability. If you’ve ever looked at Consumer appliance reviews, you know that they use and test new devices for the initial review and collect longer-term info from their subscribers who purchase for long-term reliability info. The long term info is harder to come by and frequently can’t be sorted out to specific model numbers. But there are very few sources for long-term reliability info. CR’s numbers may have some systematic bias — someone could make an argument that their source population is old and not tech-savvy, perhaps — but they are at least a reasonable effort. But no one has made that claim. Microsoft’s denial reeked of wordsmithing and hedging, with no real substance.

    More significantly, CR’s problem with Apple came down to an unfortunate interaction between their test procedure (part of their initial assessment) and a single bug. So with some agreement to share info Apple was able to resolve it. This is not the same kind of animal as customer-reported long-term reliability. CR can’t point to a detailed test procedure that was problematic; they have no first hand information. With long-term reliability there may well be multiple contributing causes. I think it is unlikely CR will retract unless they find some problem with their statistics — unlikely at best.

    As to all the negativity to CR, it’s exactly the kind of blowback they got 20 years ago for their car reviews. But auto reliability numbers have told a consistent (improving) story. I hope that there has been some fluke; MS is a decent company which has a good track record for decent quality. But look at the comments acknowledging that many enthusiastic Surface owners have had repeated reliability problems. If I am buying for a family member, I will avoid Surface products until there is evidence their reliability issues are resolved.

  5. skipaq

    Jeff, this is a very fair and balanced review of CR’s findings. I would hope they ran their findings by Microsoft before going public. It has been more than twenty years since I trusted or read anything that CR published.

  6. ChipMcK

    The nay-sayers missed that 25% of the responders REPORTED problems.
    CR is just the messenger of the news, bad or good.

  7. jimblock

    As boltar and ChipMcK said above, Consumer Reports long-term reliability testing is NOT done in house, but is based on their annual survey which is sent to all CR subscribers. So it has nothing to do with CR’s in-house testing results, (which I also often disagree with). The real issues (if any) are whether CR subscribers who respond to their survey are typical users of the products, and whether the consumer’s perceived problems are real problems or not.

    Filling out a CR survey is time consuming, and many will skip it. Those who have had real or perceived problems are more likely to respond, so this may introduce some sampling bias.

    Further, inexperienced computer users may report a “problem” when it is not a bug, but the user has not fully read instructions, or is not very computer literate. I often help friends who think there is something wrong with an Apple product, but in fact, they did not understand how it worked or how to do what they wanted to do.

    So, from the reliability rate reported by CR readers, I may infer that Surface products are either not as intuitive as other laptops/computers, or that there are more problems. Unreliability reported by Mac users is much lower, so there appears to be a significant difference.

  8. John Kheit

    Super article Jeff. It’s time this piece of crap rag died the death of irrelevance it so deserves. Heck, if mac and windows folks can come together on this, maybe our union still has a chance

Add a Comment

Log in to comment (TMO, Twitter, Facebook) or Register for a TMO Account