We had to buy a microwave oven the other day, so I did what I always do before making a purchase: I went online to Consumer Reports.org to do some research. I’m not alone: according to the Pew Internet and American Life Project, of the 79% of adult Americans who use the Internet, 81% “look for information online about a service or product (they) are thinking of buying.”
Not all of those pre-purchase researchers go to Consumer Reports, but my wife and I do, just like my parents have always done. This time, though, I was shocked by the results. (I apologize that I can’t link to the results, or that I won’t be mentioning them in this article, but CR is a subscription service, and I don’t wish to violate the terms of use.)
In their Microwave section, Consumer Reports rated various countertop microwaves from multiple manufacturers as Best Buys, and I read the rankings on all of them. Then I noticed Customer Reviews for each model — and that the Average Ratings for the top models were glaringly bad. In fact, the average reviews for all the models rated were bad. As I read the reviews, one common thread emerged: the customers, all of whom are Consumer Reports subscribers, not only disagreed with the CR ratings and reported reliability, but were genuinely upset that CR had given them information that led to an unsatisfying, and in many cases disastrous or even explosive results. I lost track of the number of “Shame on you, Consumer Reports” type comments I read.
I read all of the reviews, desperately searching for a Microwave that had a positive result. One review mentioned that they eventually found a good microwave by reading the customer reviews on Best Buy, even though the units rated well on BestBuy were not rated well on CR.
So I went to BestBuy.com, read the reviews and found three microwaves that didn’t sound like they’d blow up or die inexplicably whether within or outside of the warranty period. Then I did what Pew says most Americans who research products online do: I went to an actual brick and mortar store to make my purchase. (One side note: while in the store, I heard a Best Buy employee interacting with a shopper. When asked about the GE models, he said something like, “I’d recommend anything we sell except for the GE’s. They’ve been having quality problems for the past few years.” Now I’m not saying that CR rated GE products highly, or even at all, or that there were customer reviews on the CR website that singled out GE for quality problems; I’m simply saying what I heard in Best Buy.)
I left the store, happy and secure in the knowledge that my choice was backed up by the real experiences of real people — a feeling that I used to get from basing my choices on reviews in Consumer Reports.
Before you dismiss online customer reviews as the exclusive domain of malcontents, consider this survey by Bazaarvoice and Keller Fay, user review and WOM experts, reported here on Search Engine Watch: “…79 percent of reviewers write reviews to reward a company for the quality of the product or service they bought, with 87 percent of the reviews being positive in tone… 97 percent of review readers find the reviews they read to be accurate.”
Customer Reviews are one of the most utilized forms of consumer generated content. When it comes to buying cars, JD Power’s 2008 New Autoshopper.com Study reports that 70% of autmotive Internet users utilize consumer generated content when shopping for a car, with 63% using customer ratings and reviews. (You can download the study here.) Search Engine Watch blogs here that customer reviews are one of the most important sources of product information, second only to word of mouth from a friend.
And before you dismiss the value of Consumer Reports, please consider that they were honest enough to print customer reviews that not only disagreed with them, but openly questioned the validity of their ratings. Those reviews sent me in a direction that led to my satisfied product purchase.
Will I ever use Consumer Reports to research a product before purchase again? You betcha! CR is still a great resource for product and category information, and their testing facilities still provide data that can’t be gotten anywhere else.
Will I ever skip the customer reviews and just read the ratings? What do you think?
But this whole experience leaves me with one unanswered question: Can someone please explain to me why there is such a consistent, category-wide disagreement between the ratings of the professional researchers at Consumer Reports and of the consumers reporting their own experiences with the same products?
Can someone please explain...? 
6 replies on “Consumer Reports vs. consumers reporting”
I have used Consumer Reports to some degree in the past. But the reason I no longer use them is that almost consistently their top picks did not match other site’s top picks or people’s reviews. I don’t know whether their reviewers are just simply using the wrong testing methods or there are some back room payolas going on. What it comes down to is I no longer trust them. What I tend to use these days is CNet for electronic gear, dpreview.com for cameras and Amazon’s consumer reviews for all other items. For Amazon I look for the items that has the most reviews and among those which has the 5 stars. You still have to wade through and read the reviews.
I wonder how many other people have given up on Consumer Reports for the reasons we’re discussing? BTW, I use CNet for electronics, too, but also more importantly for software. I don’t think I’ve bought a program yet I did’nt check out on CNet first. I’ve got to check out dpreview for cameras, especially since I’m looking into getting a Flip. (I need a very portable video camera.) I use Amazon for book reviews and for other reviews, but I never thought of making them a review destination, which was silly of me because they virtually invented the concept of customer reviews, have broad range of product, and a reviewer rating system that lets you know if a reviewer is useful or not. Good comment, Restie — thanks.
Have you asked CR about the discrepancy?
I still use CR, and have had no problems. I usually read the customer reviews, but don’t give them much weight. We’ve used them for a microwave, dishwasher, window AC, and portable AC. Once or twice I’ve been surprised that they didn’t have a list for whatever I was searching for.
For the microwave, we ended up going to the store and buying whatever they had in stock that was close to the CR pick. We weren’t willing to wait for delivery.
Ah, Josh — you bring up another of my peeves with CR: finding the model they review in a store! But no, I haven’t asked CR about the discrepancy. Interesting idea. As a famous pointy-eared alternately bearded character once said, “I shall consider it.”
i also relied on CR to make a microwave purchase. boy, did i get burned. i spent around $350 for a samsung that broke down (magnetron failure) after month 15. to me that is unacceptable of any microwave, especially one that was very highly rated by CR.
so the question becomes: where does one find a good supply of consumer reviews?
Thanks for the comment, David. In this instance, I found a good variety of customer reviews on both Consumer Reports and the Best Buy site, which I checked out as the result of a suggestion by a customer review on CR! There’s always Amazon, (which Restie mentioned above) which pretty much invented the Customer Review and sells a wide variety of products. There’s also epinions.com, which I often find helpful but sometimes doesn’t have as many reviews as I’d like about a product. BizRate has reviews, too, but they didn’t have anything on microwaves. And, as Restie also mentioned, CNet, which is phenomenal for electronics. Hope this helps.