Car Buying Tips: Consumer Reports, You Decide

car buying tips consumer reports you decide
Consumer Reports has released the 2007 edition of its “Annual Auto Issue.” For the second year in a row, all CR’s “Top Picks” come from Japanese makes. For some industry observers, that’s a problem. They believe the magazine’s results indicate a hidden bias, especially against vehicles produced by domestic manufacturers. Which both is and isn’t true.

Consumer Reports’ road test engineers subjected every test vehicle to a thorough evaluation, using a pre-established set of criteria and weights. For example, emergency handling might get ten points, front seat comfort might receive eight and “feels like a Honda” might be worth 37 (just kidding— I hope). Whatever the formula, when the magazine totaled-up the points, they ended up with a list composed entirely of Japanese cars.

This process leads to an obvious question: what criteria and weights– what formula– does Consumer Reports use to rate any given vehicle? The press and Consumer Reports have a policy in this regard: don’t ask, won’t tell.

At the last Detroit auto show, I asked a Consumer Reports road test engineer why the magazine doesn’t publish its formulas. After all, nearly every enthusiast-oriented magazine does when conducting a comparison test. “It’s policy,” he replied. He went on to suggest that he didn’t make the policy, he didn’t necessarily support it, but as a Consumer Reports employee, he had no choice but to follow it.

It's time for Consumer Reports to declassify its formulas. Two days ago, someone made the same request on their forum. The moderator’s response was revealing (or not):

“Thank you for your comments. These forums are designed to help subscribers in selecting and buying a car. They are intended to be primarily peer to peer, with our Auto test experts helping out when available.“If you find errors we will be glad to look into them and make corrections, but we just don’t have the time or resources to engage in lengthy debates here.

“You can channel your inquiries through Customer Service. There is a link on the bottom of every forum page, and at the top of every CR on-line page. You are also welcome to visit our facilities when we hold an open house and speak directly with our test staff at that time.”

As an automotive data provider, I find Consumer Reports’ arrogance, intransigence and unaccountability completely unacceptable. Any company that depends on the public trust must strive for transparency. If you have nothing to hide, you hide nothing. That’s why I respond to any and all questions about TrueDelta’s methodology. Besides, engaging in open public debate can teach data providers better ways to do things.

The Consumer Reports moderator’s non-response indicates that the magazine doesn’t see how knowledge of its overall score formulas could further improve anyone’s ability to find the right car. These formulas are divulged on a need-to-know basis, and as far as they’re concerned car shoppers don’t need to know.

Sorry, but it just isn’t so. To keep things simple, let’s assume there are only two criteria, ride and handling. Let’s further assume that Consumer Reports’ editors have decided that ride quality is twice as important as handling when evaluating a minivan. Keeping the overall score formula secret implies that any reasonable minivan buyers should also weight ride quality twice as heavily as handling.

This is just plain wrong. There is no objective way to arrive at one best formula for everyone. For some minivan buyers handling is twice as important as ride quality, and there’s nothing inherently superior about either set of weights. These are necessarily subjective value judgments. The Toyota Sienna is Consumer Reports’ “top pick” among minivans. But a buyer who values handling above ride quality will be happier in a Honda Odyssey.

This is not to say that overall evaluations are necessarily useless. If the formula was provided, our minivanista could determine that giving extra weight to handling would tip the decision in favor of the Honda.

But the formula is not provided, so there’s no way for minivan shoppers to know how closely Consumer Reports’ criteria and weights match their own or how adjusting these might affect the decision. By withholding its formulas, Consumer Reports takes the stand that readers should let the Yonkers mob decide for them what matters most– and least– when choosing a vehicle.

It’s true that many, even most car buyers are intimidated by the process– to the point where they want an authority figure to tell them what to buy. But buyers truly interested in finding the best fit for their personal tastes are going to have to put forth more effort. Currently they’ll have to rely on sources other than Consumer Reports, since the magazine withholds information needed by people who want to think for themselves.

[Michael Karesh operates, a vehicle reliability and price comparison website.]

Join the conversation
2 of 91 comments
  • Automaton Automaton on Mar 14, 2007

    I don't know if this has been mentioned before, but, CR doesn't publish the criteria for their tests on ANYTHING they test because they don't want manufacturers to build to that test criteria (in the same way car manufacturers build their cars to the CAFE standards and have, as a result, made that test inaccurate). It's a means to prompting the manufacurers to build the best possible product across all perameters.

  • on Oct 16, 2007

    [...] Re: Consmer Reports update Here, read this: Car Buying Tips: Consumer Reports, You Decide | The Truth About Cars I think CR is full of crap and is succumbing to pressure from domestic automakers in an attempt to preserve what is soon to be the end of an era - a US stronghold on the light truck market - a little while longer. When I think that as my head hits my pillow tonight, I will sleep just just fine. I hope they wake up and smell the coffee. My 2 cents. __________________ 2007 DC LTD, Silver Metallic. MODS: Line-X Bedliner; Toytec 3" Lift; HUGE Moto Metal 20" Wheels and 35*13.5*20 Procomp Xtreme A/T Tires; Sirius Radio; Nuvo GPS Navigation; Hornblasters; Black Westin Bull Bar with fog lights; Northern Tools Tool Box; TRD CAI; Color-matched Fender Flares; Boorla Catback Dual Exhaust. ***Pics in gallery. [...]

  • Johnster Not feelin' it. The traditional unreliability of turbo engines is a big turn-off, especially in a work truck that (I hope) you'd want to keep on the road for 200,000 miles or more without having major repairs.
  • ToolGuy Car audio is way overpriced.
  • Marty S The original Charger was a 2 door, as was the landmark 68 model. Its funny that some younger commenters are surprised that its not a four door. I never understood why modern Chargers have been four door sedans. I think the best looking Charger was the 68, absolutely perfect in its lines and proportions. This concept really emulates that and I think I think it looks great.
  • Master Baiter The D-bag elites like Al Gore demanding that we all switch to EVs are the type of people who don't actually drive. They get chauffeured around in black Yukon Denalis. Tesla does have a good charging network--maybe someday they will produce a car that doesn't suck.
  • MRF 95 T-Bird As a Challenger GT awd owner I lIke it’s heritage inspired styling a lot. There’s a lot of 66-67 as well as 68-70 Charger in there. It’s refreshing that it doesn’t look like a blob like Tesla, Volt/Bolt, Mach-e BMW I whatever etc. The fact that it’s a hatch makes it even better as a everyday driver thus eliminating the need for a CUV. If it’s well built and has a reliable track record I can see trading up to it in a few years.