Car Buying Tips: Consumer Reports, You Decide

Michael Karesh
by Michael Karesh
car buying tips consumer reports you decide
Consumer Reports has released the 2007 edition of its “Annual Auto Issue.” For the second year in a row, all CR’s “Top Picks” come from Japanese makes. For some industry observers, that’s a problem. They believe the magazine’s results indicate a hidden bias, especially against vehicles produced by domestic manufacturers. Which both is and isn’t true.

Consumer Reports’ road test engineers subjected every test vehicle to a thorough evaluation, using a pre-established set of criteria and weights. For example, emergency handling might get ten points, front seat comfort might receive eight and “feels like a Honda” might be worth 37 (just kidding— I hope). Whatever the formula, when the magazine totaled-up the points, they ended up with a list composed entirely of Japanese cars.

This process leads to an obvious question: what criteria and weights– what formula– does Consumer Reports use to rate any given vehicle? The press and Consumer Reports have a policy in this regard: don’t ask, won’t tell.

At the last Detroit auto show, I asked a Consumer Reports road test engineer why the magazine doesn’t publish its formulas. After all, nearly every enthusiast-oriented magazine does when conducting a comparison test. “It’s policy,” he replied. He went on to suggest that he didn’t make the policy, he didn’t necessarily support it, but as a Consumer Reports employee, he had no choice but to follow it.

It's time for Consumer Reports to declassify its formulas. Two days ago, someone made the same request on their forum. The moderator’s response was revealing (or not):

“Thank you for your comments. These forums are designed to help subscribers in selecting and buying a car. They are intended to be primarily peer to peer, with our Auto test experts helping out when available.“If you find errors we will be glad to look into them and make corrections, but we just don’t have the time or resources to engage in lengthy debates here.

“You can channel your inquiries through Customer Service. There is a link on the bottom of every forum page, and at the top of every CR on-line page. You are also welcome to visit our facilities when we hold an open house and speak directly with our test staff at that time.”

As an automotive data provider, I find Consumer Reports’ arrogance, intransigence and unaccountability completely unacceptable. Any company that depends on the public trust must strive for transparency. If you have nothing to hide, you hide nothing. That’s why I respond to any and all questions about TrueDelta’s methodology. Besides, engaging in open public debate can teach data providers better ways to do things.

The Consumer Reports moderator’s non-response indicates that the magazine doesn’t see how knowledge of its overall score formulas could further improve anyone’s ability to find the right car. These formulas are divulged on a need-to-know basis, and as far as they’re concerned car shoppers don’t need to know.

Sorry, but it just isn’t so. To keep things simple, let’s assume there are only two criteria, ride and handling. Let’s further assume that Consumer Reports’ editors have decided that ride quality is twice as important as handling when evaluating a minivan. Keeping the overall score formula secret implies that any reasonable minivan buyers should also weight ride quality twice as heavily as handling.

This is just plain wrong. There is no objective way to arrive at one best formula for everyone. For some minivan buyers handling is twice as important as ride quality, and there’s nothing inherently superior about either set of weights. These are necessarily subjective value judgments. The Toyota Sienna is Consumer Reports’ “top pick” among minivans. But a buyer who values handling above ride quality will be happier in a Honda Odyssey.

This is not to say that overall evaluations are necessarily useless. If the formula was provided, our minivanista could determine that giving extra weight to handling would tip the decision in favor of the Honda.

But the formula is not provided, so there’s no way for minivan shoppers to know how closely Consumer Reports’ criteria and weights match their own or how adjusting these might affect the decision. By withholding its formulas, Consumer Reports takes the stand that readers should let the Yonkers mob decide for them what matters most– and least– when choosing a vehicle.

It’s true that many, even most car buyers are intimidated by the process– to the point where they want an authority figure to tell them what to buy. But buyers truly interested in finding the best fit for their personal tastes are going to have to put forth more effort. Currently they’ll have to rely on sources other than Consumer Reports, since the magazine withholds information needed by people who want to think for themselves.

[Michael Karesh operates, a vehicle reliability and price comparison website.]

Join the conversation
2 of 91 comments
  • Automaton Automaton on Mar 14, 2007

    I don't know if this has been mentioned before, but, CR doesn't publish the criteria for their tests on ANYTHING they test because they don't want manufacturers to build to that test criteria (in the same way car manufacturers build their cars to the CAFE standards and have, as a result, made that test inaccurate). It's a means to prompting the manufacurers to build the best possible product across all perameters.

  • on Oct 16, 2007

    [...] Re: Consmer Reports update Here, read this: Car Buying Tips: Consumer Reports, You Decide | The Truth About Cars I think CR is full of crap and is succumbing to pressure from domestic automakers in an attempt to preserve what is soon to be the end of an era - a US stronghold on the light truck market - a little while longer. When I think that as my head hits my pillow tonight, I will sleep just just fine. I hope they wake up and smell the coffee. My 2 cents. __________________ 2007 DC LTD, Silver Metallic. MODS: Line-X Bedliner; Toytec 3" Lift; HUGE Moto Metal 20" Wheels and 35*13.5*20 Procomp Xtreme A/T Tires; Sirius Radio; Nuvo GPS Navigation; Hornblasters; Black Westin Bull Bar with fog lights; Northern Tools Tool Box; TRD CAI; Color-matched Fender Flares; Boorla Catback Dual Exhaust. ***Pics in gallery. [...]

  • Fred I owned a 2001 MR2 for 15 years nothing ever went wrong with the vehicle. It was always exciting to drive most people thought it was a boxster. The only negative was storage and legroom considering I'm a little over 6:4 the only reason it was sold was as a second car and a grandchild on the way we needed something more practical.
  • V16 I'm sure most people could find 155,365 reasons to choose another luxury brand SUV and pocket the difference.
  • ChristianWimmer I don’t want this autonomous driving garbage technology in any car.My main fear is this. Once this technology is perfected, freedom-hating eco hysterical governments (crap hole Germany, UK and the European Union in general) will attempt to ban private car ownership because “you don’t need to own a car anymore since the car can come to you, drop you off and then proceed to service the next customer”... no thanks. Having your own car is FREEDOM.Go away, autonomous driving. I also enjoy the act of driving a car. I want to drive, not be driven.
  • Mike-NB2 The solution is obvious here. Everyone should be raised in an Irish Catholic family and then all it takes is a sideways glance from mom and you're atoning for that sin for the rest of your life. My mother has been dead for decades and I still want to apologize to her. Catholic guilt is a real thing. 😁
  • Wjtinfwb A good car. I don't find Accord's as appealing as they were a decade or two ago, not that they've gotten worse, but the competition has gotten better. It would be my choice if I had to pay for it myself and maintain it for 10 years and 150k miles. They'd be very reliable and no doubt inexpensive miles, but probably a pretty boring 10 years.