Consumer Reports Announces Reliability Data; Not a TTAC Fan

consumer reports announces reliability data not a ttac fan

After David Champion, head of Consumer Reports‘ auto testing, presented this year’s reliability results, I asked two simple questions. 1. What month were most surveys returned (i.e. how old are the data)? 2. What problem rates do the dots represent? Or, to keep it as simple as possible, what was the average problem rate for a 2008 car? Unfortunately, Mr. Champion did not know the answer to either question. He could only respond that the surveys went out in “the springtime,” and that the dots are relative. As if the actual problem rates they represent were of no consequence. In fact, both things matter. The truth about CR, as we’ve noted here: before: 1. The data are already about five months old, and will be 17 months old before they are updated again. 2. The differences between the dots for a 2008 model are about one problem for every thirty cars. But, since even the head of CR’s auto research doesn’t know these facts, it should come as no surprise that their millions of subscribers haven’t a clue. And then things got ugly…

Afterwards, CR’s head of publicity came up to speak with me, and was very combative. He insisted that I retract a recent statement that CR’s data were 17 months old. My response: weren’t they 17 months old when I wrote that? Well, yes, but not anymore. Won’t these new data also be 17 months old before they’re updated? Well, yes, but… He then challenged that my turnaround would be no better if I had 1.4 million responses. You know, because its better to be large than to be up-to-date. Didn’t the Detroit auto companies wear out this argument?

Finally he brought up my comment on David Holzman’s piece about the future car conference CR hosted recently. My comment: “CR avoids corruption itself, but corrupts other media as much as possible.” Or something to that effect. My response: didn’t we just enjoy a nice meal, preceded by an open bar? Weren’t all of the questions other than mine easy? (Actually, mine should have been easy, to answer at least.) His only response: there were tough questions. But there weren’t. And I wouldn’t be surprised if we’re not invited back next year.

[CR paid for MK’s parking, lunch and the two ginger ales from the open bar. Fair disclosure: TTAC has a contract with MK’s TrueDelta for pricing and specification data]

Join the conversation
2 of 57 comments
  • Noreserve Noreserve on Oct 24, 2008

    I have been a CR subscriber for many years. I respect their independence from advertising. That doesn't make them perfect. No one claims that. I think they do a great job overall on a wide variety of product testing. They have also admitted when they screw up - car seat testing results from an outside contractor miscommunication comes to mind. They do seem to have integrity in their mission and in their testing methodologies. Mr. Champion didn't know what month most of the surveys were returned - big deal. Maybe the more appropriate question would have been "How old, on average, are the returned survey results?" Even then, we all know that this is pen/paper mail stuff here. Yes, is going to be a delay. I would guess that there are those who return the surveys the next day and those who get around to it several months later. Either way, they probably average out to some number. So he didn't have the specific month. Did that mean he couldn't get that info? Probably not. You say that their head of publicity was combative. Maybe we should also hear their side of the story. Your second question about dot-to-problem rate correspondence is a good one. I would like to see their answer to that one. Perhaps your questions could be communicated via e-mail to them and their answers posted here. I'm not going to be a blanket apologist for CR here. Politics and "stick to the goddamn toaster testing" comments aside, they do a commendable job in bringing information to me and millions of others who would otherwise be in the dark. Their work on safety testing in multiple areas (auto safety, lead, radon, et al) is enough in my book to have a permanent place on my reference shelf. They are a tireless advocate in these areas. They have a tremendous amount of respect. That same respect would not be there if they had advertising of any sort. And claiming on a website that "our opinions are in no way influenced by those ads floating next to that shit you're trying to read" doesn't cut it and everyone knows it. Yes, you guys are competitors in a limited sense. I really like the TrueDelta concept and website. I'd like to see a lot more problem detail. And maybe some info as to what doesn't match up to CR and others. I wish it could happen without ads though. For example, having a Warranty Direct ad on your page removes your ability to objectively discuss extended warranties - a very relevant subject in your realm. I have always laughed when I read some of the forum crap from people dismissing CR auto testing. They do a pretty damn thorough job from what I've seen over the years. And they have their own track, buy their own vehicles, etc. Their summaries of most models are usually spot-on. They get hit, at least perceptually, for their board-stiff engineers assessment of things like sports cars. They usually hit the nail on the head though. I've always found it odd that the major rags have never given a shit about listing reliability for vehicles - aside from a handful of "long-term tests". And these are cars that are usually "provided" to them, not purchased like CR's policy. And safety results. How many rags have a damn bit of useful info on safety? Does C&D or Automobile give a mention to how safe this gleaming sled is? Usually not anywhere to be found. Guess they figure it's a given that we'll get it elsewhere. Overall, I'd recommend treading lightly on the CR criticism until you provide us with their official response so that we can evaluate in black and white.

  • Michael Karesh Michael Karesh on Oct 24, 2008

    I actually thought that asking which month the surveys were returned--and 75% are online these days--was a less loaded question than "how old are the data," which would imply that the data are old. Maybe I was wrong in this, but I thought it was a very simple question. Being in the survey business myself, there are few things I focus more on than how many surveys come in how soon. I can tell you that the current round is tracking almost exactly with the previous round. Maybe when you have 1.4m of them, though, you no longer care how many actually get returned, or when. I don't currently evaluate extended warranties. If I did, then having Warranty Direct ads would pose a conflict. Currently I don't personally evaluate anything on the site itselft--the content is user-generated. My personal material is posted here and elsewhere.

  • Inside Looking Out Why EBFlex dominates this EV discussion? Just because he is a Ford expert?
  • Marky S. Very nice article and photos. I am a HUGE Edsel fan. I have always been fascinated with the "Charlie Brown of Cars." Allow me to make a minor correction to add here: the Pacer line was the second-from-bottom rung Edsel, not the entry-level trim. That would be the Edsel Ranger for 1958. It had the widest array of body styles. The Ranger 2-door sedan (with a "B-pillar", not a pillarless hardtop), was priced at $2,484. So, the Ranger and Pacer both used the smaller Ford body. The next two upscale Edsel's were based on the Mercury body, are were: Corsair, and, top-line Citation. Although the 1959 style is my fav. I would love a '58 Edsel Pacer 4-door hardtop sedan!
  • Lou_BC Stupid to kill the 6ft box in the crewcab. That's the most common Canyon/Colorado trim I see. That kills the utility of a small truck. The extended cab was a poor seller so that makes sense. GM should have kept the diesel. It's a decent engine that mates well with the 6 speed. Fuel economy is impressive.
  • Lou_BC High end EV's are selling well. Car companies are taking advantage of that fact. I see quite a few $100k pickups in my travels so why is that ok but $100k EV's are bad? The cynical side of me sees car companies tack on 8k premiums to EV's around the time we see governments up EV credits. Coincidence? No fooking way.
  • EBFlex "I'd add to that right now, demand is higher than supply, so basic business rules say to raise the price."Demand is very low. Supply is even lower. Saying that demand is outstripping supply without providing context is dishonest at best.