Unraveling The Mystery Of Consumer Reports' Brand Spread

Michael Karesh
by Michael Karesh

Some things have been repeated so often that many people have come to accept them as facts. I tripped across one of these in Bob Lutz’s new book, Car Guys vs. Bean Counters (review on the way). Lutz offers “a curiosity I have observed several times at various stages of my career”: when the domestics rebadge an import, the resulting model has scored “way lower” in Consumer Reports reliability survey. This has been Exhibit A in the argument, also repeated by Lutz, that import owners under-report problems on surveys in order to “retroactively justify the wisdom of their purchase.” I’ve come across this claim about CR so many times in the past that it just had to be true. Then I checked.


The cars Lutz mentions: Eclipse / Laser, Corolla / Prizm, and Matrix / Vibe. I happen to have some Consumer Reports annual auto issues from the years in question. In the 1992 issue, the Plymouth Laser and Eagle Talon sometimes scored lower than the Mitsubishi Eclipse, and sometimes scored higher, depending on the model year and powertrain. The differences are mixed and never substantial. CR pooled the samples for the Diamond Star hatches beginning with the 1993 issue, so the scores for the triplets were absolutely identical from that point on.

Also in the 1992 issue, the Prizm scored lower than the Toyota Corolla, but just by a little, not “way lower” as Lutz states. In the 1995 and 1996 issues the Prizm actually scored higher than the Corolla, with the margin in the former year a substantial ten points.

With the Vibe and Matrix, Lutz gets his snark on. “Have you been paying attention? Test question: which of the ‘twins’ performed better in quality surveys?” Well, in the 2005 issue the Pontiac Vibe and Toyota Matrix had virtually the same scores, with both better than average. In the 2009 issue the Matrix scored a little higher than the Vibe, but in the 2007 and 2008 issues the reverse was true. In fact, in the 2008 issue the Vibe had the highest score in the entire “wagons and hatchbacks” category. It seems that Lutz either wasn’t paying attention or saw the unfair playing field he wanted to see, didn’t check the facts either way (always a good idea), and consequently failed his own test.

Michael Karesh operates TrueDelta, an online source of automotive pricing and reliability data.

Michael Karesh
Michael Karesh

Michael Karesh lives in West Bloomfield, Michigan, with his wife and three children. In 2003 he received a Ph.D. from the University of Chicago. While in Chicago he worked at the National Opinion Research Center, a leader in the field of survey research. For his doctoral thesis, he spent a year-and-a-half inside an automaker studying how and how well it understood consumers when developing new products. While pursuing the degree he taught consumer behavior and product development at Oakland University. Since 1999, he has contributed auto reviews to Epinions, where he is currently one of two people in charge of the autos section. Since earning the degree he has continued to care for his children (school, gymnastics, tae-kwan-do...) and write reviews for Epinions and, more recently, The Truth About Cars while developing TrueDelta, a vehicle reliability and price comparison site.

More by Michael Karesh

Comments
Join the conversation
2 of 74 comments
  • Jim1961 Jim1961 on Sep 10, 2011

    The one major flaw in this article is there are no numbers given except when an American branded car beat it's twin. Why are there no numbers given? I'm no expert in statistics but I know that margin of error becomes lower as the sample size increases. If the difference between two identical cars with different brands is outside of the margin of error then bias is likely the cause. How far outside of the margin or error were the results? Without hard numbers to back up their claims Bob Lutz's AND Michael Karesh's assertions are both garbage. Another interesting bit of information brought up was that CR merged the results of cars that are twins or triplets. I'm not a conspiracy-minded person but I have to wonder if it's possible CR did this in part to avoid the controversy regarding the bias of consumer union members. Here are some observations of mine that are not backed up by numbers but are hard to ignore. Just the fact that "twin" cars such as the Corolla and Prism SOLD in vastly different numbers shows bias of consumers. Beginning in the 1980s Japanese brand cars did very well in reliability according to Consumer Reports. I don't dispute this at all but not all Japanese car makers are equal. In Consumer Reports Mitsubishi ranks very, very low yet car buyers keep purchasing Mitsubishi cars because they think all Japanese cars are superior. Another misconception is that Japanese cars are ranked number one in consumer reports and American cars ranked worst and European cars are in the middle. According to consumer reports it's European cars that are the most unreliable and American cars were in the middle. A friend of mine purchased a Volkswagen Toureg which is one of the most problematic vehicles according to Consumer Reports, J.D. Power and Associates and my friend, David, who warns everyone he knows not to buy a Volkswagen.

  • Brobdingnagian Brobdingnagian on Sep 12, 2011

    This theory can be taken a step further: As a BMW and Benz owner, I am typically obsessive and likely to notice and complain about unsatisfactory details. I have always suspected this is why these brands don't typically score so high on Consumer Reports surveys and such. A typical Camry owner, by contrast, is just happy to have an appliance that starts and rolls to its destination...ergo, few complaints. I have always been happy and impressed with the BMWs and Benzes, and willing to have necessary maintenance done on them, but my detail orientation allows me to see the most minor flaws. My expectations are higher. Bottom line: Consumer Reports is crap when it comes to cars, as we all know.

Next