By on March 2, 2011

Wondering why CR picked the cars it did for its recent “Top Picks”? Are you curious where it got reliability data for brand new cars like the new Elantra? Do you wonder how a “classic” like the Avalanche won top pick for trucks? Get the truth from the horses mouth, by checking out CR’s VYou page, where you can submit your burning questions and receive video responses. Accountability, here we come!

Get the latest TTAC e-Newsletter!

38 Comments on “Got Questions About Consumer Reports Car Picks?...”


  • avatar
    MRL325i

    I stopped paying attention to CR when they became just another cog in the liberal MSM gearset.  They always were, but once they started shilling for Obamacare et al I just tuned out and let my subscription lapse.

    • 0 avatar
      psarhjinian

      Not that I’m not itching to get into (yet another painfully redundant) health-care dust-up on the internet, but what does supporting socialized medicine (or not) have to do with their ability to present reliability and performance statistics on cars, washing machines, televisions and so forth?

    • 0 avatar
      John Fritz

      Is it possible that demonstrating support of the destruction of health care by the federal government means they might also skew data to support government sponsored car companies? Stranger things have happened.

    • 0 avatar
      psarhjinian

      Considering that they still kick the stuffing out of Chrysler and GM and gave nods to their competition, that seems rather unlikely.
       
      CR isn’t really in favour of Obamacare, per se.  They did come on side of effective single-payer health care the likes of which most real left-wingers wanted and that the rest of the Western world has, rather than the barely-an-improvement milquetoast debacle that actually passed.

    • 0 avatar
      Morea

      About 15 years ago CR made an explicit decision to move from being solely a testing lab to being a consumer advocacy group.  So be it, it’s their organization.

      But the US could use more independent, non-profit product testing labs, that is, organizations that are solely numbers/fact based and leave the consumer to decide what is best.

  • avatar
    Autojunkie

    Their data gathering process is flawed…

    CR has become irrelevant.

    • 0 avatar
      psarhjinian

      And has been replaced by….  What?  Anecdotes on blogs?  Trending in Twitter?
       
      Yes, it’s flawed.  So is democracy.  Find me a better system.

    • 0 avatar

      Here’s your better system:

      http://www.truedelta.com/car-reliability.php

      As much data? No, but we’ll get there. Quality vs. quantity for now.

      As for the Q&A, if it’s like the past they’ll be happy to answer questions about *cars* but will dodge those about their *methods.* Notice that they’re soliciting “car questions.” Their methods are assumed to be beyond question and they tend to respond either “we’ll get back to you on that” (but never do) or “it’s proprietary.”

    • 0 avatar
      Mr. Sparky

      I’m still boycotting TrueDelta over Mr. Karesh’s two part article questioning Consumer Reports reliablity data.

      Once TrueDelta becomes a little less defensive about CR, perhaps I’ll reconsider, but until then, I’ll stick with the non-profit organization that’s been around since 1936.

    • 0 avatar
      psarhjinian

      @Michael: I take your point and (more or less) agree with you, but if someone doesn’t trust CR because it’s collection methods are ostensibly flawed, then logically, by extension, your site is flawed for the same reason.
       

    • 0 avatar
      Silvy_nonsense

      “Here’s your better system”
       
      That’s debatable.
       
      True Delta’s user interface is abysmal to the point of making the tool useless. True Delta has so few cars in the database that it’s barely relevant.
       
      If you want to be the best auto reliability data source on the web, you’ve got a massive amount of work ahead of you. Hopefully, you’ll get there, but until then, stop comparing True Delta to Consumer Reports. At this point, you’re not even close to being in their league.
       
      Knocking your competition is weak and pitiful. If you can’t sell your product on its own merits, that’s a pretty good sign you don’t have much of a product. Man-up, fix your crummy interface, get several hundred thousand more cars in your database and then get back to us.
       
      Note to the editor: When one of your journalists uses the site frequently and aggressively to push his industry-related product in both stories and comments, it undermines all the talk about TTAC providing the unvarnished truth, without outside influence or bias. Karesh in an industry player and every time Karesh publishes a story or comment, especially one regarding vehicle reliability, its logical to assume that he’s slanted it in favor of portraying his commercial product in a favorable light. It’s an obvious conflict of interest and it makes TTAC seem just as slanted and unworthy of my trust as the rest of the automotive press. Its time for you to take a serious look at your editorial policies regarding conflicts of interest and take some decisive action.

    • 0 avatar
      iNeon

      +1 for Silvy

    • 0 avatar
      Fugue

      I rarely agree with silvy.. but he does have a point. I have encouraged others to join the true delta network and have noticed a steady theme of complaint about the user interface. Just my honest feedback. But i do think you are after some great and want to see it succeed. Thats why I still contribute my data.

    • 0 avatar
      Bucknasty

      Long time reader….first time commenter.  Have to agree with Silvy_nonsense and the credibility issue.  The constant True Delta plugs definitely undermine the credibility around here.  Also, CR isn’t perfect (or even necessarily the best), but I am happy it is around so I can balance what I read there with other information from other sources to help form my opinion.

    • 0 avatar

      I’m not defensive. I am frustrated. I’m beyond ready for people to drop their blind faith in CR and take a real look at their methods and results. Once this happens I’ll be quite happy to never say another word about CR. But while this blind faith remains prevalent it is very hard to offer a superior alternative, as people simply aren’t looking for one. They don’t realize that one is needed.

      No research is perfect. And I’m certainly not advocating that a survey doesn’t provide useful information unless its methods are perfect. But flaws differ in severity. Even CR’s information can be very useful, but it would be much better if they corrected some substantial flaws in their methods.

      As for bias, I invite anyone to point to an actual instance where something I’ve said would not have been said by someone without a stake in the matter. I understand, and regret, the appearance of bias. But there is no substance to it, which is why Ed permits me to write what I do.

      On the user interface, I’m very sensitive to these complaints, and very much want to do something to fix the issues. The lack of feedback specific enough to act upon hasn’t helped. I always request such feedback, but hardly ever receive it. The site is about to be redesigned, and the designer feels that the new design is much more usable. I’m not so sure, if only because it’s still not clear to me what the specific problems are. I expect to have to keep improving this aspect of the site.

    • 0 avatar
      vbofw

      “Knocking your competition is weak and pitiful”
       
      You have a couple of good points, but they guy offered a topical response to a subject being debated, and you turned it nasty.  Take a read through the comments.  I’m sure most will “knock” CR.
       
      Your opus to the Editor makes a giant leap, somehow connecting a guy’s side project of tracking reliability stats, to the objectivity of a site which reviews cars and comments on industry developments.  It’s not like Karesh has ties to the OEMs.  Like I said I think you have a couple good points but much appears misguided, and you turned it nasty by calling his actions weak and pitiful.  Tell Steve Jobs that knocking the competition is weak and pitiful.

    • 0 avatar

      Off the top of my head, here are some interface-related improvements TrueDelta could use:

      – make the search forms submit with GET requests so that you can use the back buttons properly throughout the site.  POST should only be used when you’re collecting data.
      – Don’t pass the session ID around in every page request.  Use cookies to store the session ID so that the system’s better able to keep track of whether or not you’re logged in, regardless of how you get linked to the site.
      – Allow a user to compare specific cars of their choosing

    • 0 avatar

      Thank, Aaron, this is exactly the sort of feedback I’m looking for.
       
      The first one I was totally unaware of, and will take a closer look at.
       
      We started using cookies a few months ago. I’ve left the old session variables in place, though, for those visitors who disable cookies. Surprisingly, this still seems to be 10-15% of the total.
       
      A few other people have suggested making comparisons between two cars easier, and this is something I plan to implement in the next few months. I’d do it sooner, but doing it right requires some major changes to the database.

    • 0 avatar

      @Michael:
       
      No problem.  I definitely want to see TrueDelta succeed.
       
      I see now that you do in fact use cookies but your session max lifetime must be set pretty low (maybe 20 mins?) because if I’m inactive on the site for any period of time and come back it’s forgotten who I am.  That’s frustrating.
       
      The site’s also a little confusing about letting you know if you’re logged in.  The homepage shows a “click to join” button quite prominently even if you’re already logged in.  That’s confusing.  I feel like there are other places it does the same thing but can’t locate them right now.

    • 0 avatar
      NulloModo

      I’m no fan of CR because I’ve seen how wrong their recommendations are a lot of the time.  The CR results are always going to be skewed my the demographics of the CR survey responders, which in the case of their automotive reviews do not tend to include enthusiasts.
       
      TrueDelta is a resource to get some information from, but I wouldn’t treat it as gospel either.
      Really, for someone looking to make an intelligent purchase on a car, the best course of action is to go out and drive everything you can that is similar to what you may be looking for, actually talk to some knowledgeable salespeople that might be able to make some suggestions for you based on what you are looking for, go home and do your research on the forums for the makes and models of the vehicles that have made it to your short list (keeping in mind that forums do tend to get overrun a bit with those who have a problem, show up to find out how to fix it or bitch, and then disappear never to be heard from again), get some quotes from some dealers who sell the model you are looking for, and then go in and negotiate your best price from whoever you liked the best.  Yes, it takes some time, energy, and legwork, but a car is a major purchase that happens only every few years for most people, and it’s worth doing the work yourself rather than trusting the survey results of some bored housewives and retirees.

  • avatar
    mikey

    Full disclosure….I have no ties to the dude,other than being a member. IMHO Michael Karesh’s True Delta is far more “real world” than CR will ever be.

    • 0 avatar
      Zackman

      TrueDelta is growing in stature, but it will be several more years before it becomes a force to be reckoned with. It’s all about stats and sheer numbers – the numbers TD hasn’t got – yet. I’m signed up.

    • 0 avatar

      Once the sample sizes get bigger, Karesh will absolutely blow CR out of the water.  Just my heavily biased opinion, ‘natch.

    • 0 avatar

      I’m also a TrueDelta member and am optimistic about its potential.  Yes, as mentioned above, it has very real interface problems and some limitations in the # of models covered (though I haven’t bumped into any huge limitations here in the cars I’ve been researching), but that’s stuff that can be worked out in time.

      The main issue I’ve had with TrueDelta is that the primary statistic it tracks is # of repairs.  Unfortunately this seems to treat a complete engine failure just as seriously as a squeaky dash.  More important to me is the overall cost of ownership and risk of something big going wrong.  This could possibly be somewhat captured by rolling up the repair costs (which are tracked, but which are put at $0 for warranty work so still not that helpful for newer cars which are possibly very unreliable but just haven’t had any big out-of-warranty expenses yet).

      Probably the best solution I’ve thought of (though possibly prohibitively labor-intensive) would be for Karesh to sift through the reports for a given model and identify recurring problems, then put those problems in, perhaps, 3 categories: Severe, Moderate, and Minor.  You could then see at a glance that a car has 2 severe recurring problems, and 5 minor ones, (maybe even with percentages next to the issues that show the % of respondents who experienced the problem) and click through to see what those problems are and read the individual user-submissions regarding those problems.  I don’t mind taking a risk on a car that frequently gets interior squeaks, but I’m rather scared of the HPFP in a current TDI VW, and I should be able to see that, wow, about 3% of TDIs required an $8000 repair! (3% is a made up #, but, seriously, if you’re thinking of buying a new TDI you should read up on this issue)
       

    • 0 avatar

      Aaron Z,
      It’s already possible to click through from the stats to view the reported problems. I’ve also sometimes noted these in comments.
      Three months ago I added a more sophisticated problem severity question to the survey. Since the analysis looks at the past year it’ll be another nine month before we can use this new question in the analysis.
      I’d love to include repair costs. Two problems:
      1. Warranty repairs
      2. Too many respondents provide obviously incorrect repair cost figures; with a survey it is very hard to reliably gather information beyond the most obvious and basic

    • 0 avatar

      Thanks for the response, Michael.
       
      Yes, of course you’re right that you can click through to read the specific survey responses, and the rollover notes you’ve added are handy (though few and far between).
       
      Unfortunately proper comparison between vehicles remains labor-intensive, requiring careful reading of all survey responses.  If you’re trying to see the relative reliability of a certain vehicle it can’t really be ascertained at a glance.  At a glance I would hope I could easily tell that a car is a ticking time-bomb (i.e., a TDI VW) vs. just a rattle-bucket (like many Mazdas).  Right now that’s not really very feasible.
       
      Also: as best I can tell the sliding scale you use that shows where a model falls on the fewest-to-most repair scale is a bit flawed.  The top end of the scale can be terribly distorted by a single model with MANY problems.  This makes all other models that year skew towards the “fewest” part of the scale.  A scale based on what percentile the car fell in (i.e., “fewer problems than 75% of the cars manufactured this year”) would probably represent something closer to what I’m expecting from the scale.  You still have the absolute # of problems on the page for those who wish to compare those #’s.
       
      I don’t mean to trivialize the difficulty of what you’re doing.  There’s no perfect way to measure these things and I think you’re currently providing probably the most useful service of this type out there.

    • 0 avatar

      The way the scale has been set up no one model can distort it in the way you describe. Partly because we don’t cover nearly every model yet I look across model years as well as within model years to determine the end values of the scale.
       
      I also intentionally set the scales up so that few models end up at the high end, based on the logic that only the worst cars these days are so unreliable that they should be avoided on this basis alone.

  • avatar
    Zackman

    They pretty much do what we all do. Why do former Toyota owners keep buying Toyotas? Their past record. Why aren’t people buying Chryslers? Same reason. Why are the GM “W” bodies so popular? Same reason. Why will I probably buy another Impala – if they are still available? My great (so far) experience with mine. Reputation!

    • 0 avatar
      Bunter1

      Actually consumers aren’t buying the W body anymore.  The Imp was 72% fleet last year-a paltry 47000 went to retail and those all had big incentives on the hood.

      Just a thought.

      Bunter

  • avatar
    obbop

    There must be a usable correlation to be had via the make/model of “yard ornament vehicles” in the front yards of the Hillbilly hovels hereabouts atop those concrete blocks and purchaser popularity or some other aspect regarding vehicles.

  • avatar
    ajla

    Maybe I should ask some questions.  I really disagree with about half of their picks this year.

  • avatar
    Zombo

    CR is just one source of info for vehicle and other products . The smart consumer uses ALL resources to make well -informed purchases . These include the rival Consumer Guide , item specific forums on the internet , sites like True Delta , Car Survey  , Edmunds , consumer reviews on retail sites , and so forth . Stubbornly ONLY sticking to one source which has been around since the 30s is very narrow minded as I see posts on internet forums all the time blasting CR for giving them bum buying advice . I have seen both good and bad products recommended by CR over the years so using more than one resource is the only way to go .
     
    http://www.hackettstownlife.com/forum/251068

  • avatar
    zeus01

    I finds CR’s info both detailed and reasonable accurate (certainly more so than info provided by most competing publications, which often leave me wondering who paid them off). But there’s another publication besides CR that I consider to be among the more reliable sources of relatively unbiased info that doesn’t seem to get much press here: Lemon Aid. Two things I like about these rags right away:

    1. The author, Phil Edmonston, is not a car enthusiast and not easily impressed by flash and dash. Like those at CR, he sees cars as appliances rather than status symbols. A car that’s a mechanical nightmare isn’t going to score points with him just because it has awesome lines.

    2. Not only does Lemon Aid highlight the number of issues a vehicle may be cursed with (and NOTHING escapes his wrath, not even Honda and Toyota), it also delves into the exact nature of said issues, including service bulletins, secret warranties, recalls and, just as important, how the manufacturers handle their customer’s vehicle complaints post-sale.

    I may collect a few old classics that are known high-maintenance items (1st-gen RX7s, Fiat X1/9s, etc.) if they’re just for weekend drives during the summer and can be picked up for chump change say, under 5k. But my daily driver had better start every time, rarely even have a hiccup let alone break down, get good fuel economy, have slower-than-average depreciation, provide good utility and provide decent if not outstanding handling. If it’s also a hot looking car, bonus. But that’s nowhere near the top of my priority list. Therefore, the findings of someone who sees cars as appliances rather than toys carry more weight with me. Findings like those in Lemon Aid.  

    • 0 avatar
      psarhjinian

      Lemon Aid is very good.  A little extreme at times, and I’m sure that if you have issues with Ralph Nader you’ll hate Phil Edmonston’s guts, but he and the APA do good work.  It’s invaluable if you’re going to go toe-to-toe with a dealer or automaker.
       
      Side note: I actually came to TTAC because of a quote/reference printed in Lemon Aid.

  • avatar
    ihatetrees

    True Delta’s methodology is more transparent than CR’s. That said, western society is largely now math moronic. Most don’t understand that significance. And CR’s data set, however flawed, is vast.
     
    What would really benefit consumers would be the actuarial data behind extended warranty rates by model / make / year.
    Although I bet access to such info is locked-down by insurance and parts industries better than the government’s Area-51 Aliens.

  • avatar
    nrd515

    I used to get CR years ago, but after reading their sometimes seriously out of whack recommendations on all kinds of appliances and electronics, I came to the conclusion that if they are so off on that stuff, why would cars be any better?

  • avatar
    Junebug

    I agree that CR is NOT the last word in product reviews. Recently,I’ve used them along with other sources to try to determine the best washer-dryer to buy. The best resource was a local guy that repairs them. He works on all bands and tracks the changes in ownership of these companies along with bad designs, cost cutting, etc.

    Same with lawn mowers, weed trimers, if you’re a home owner and want something that lasts, ask a couple lawn care companies what they use. You pay more for pro quality but you get years of reliable service in return, plus if you need to sell it – a lawn care co will gladly buy your lightly used equipment cause they know quality.

Read all comments

Back to TopLeave a Reply

You must be logged in to post a comment.

Recent Comments

  • brn: tonycd, My previous tires were Continental PureContact. My most recent tires are an updated version of...
  • brn: None of this means jack about human caused global warming as long as we keep increasing the number of humans....
  • mcs: With Korea, it depends on the company. Actually, that applies to every country. I’m not a fan of LG. I had...
  • bullnuke: Tesla is successful because it is the iPhone of automobiles – gotta have it to be seen with it and I...
  • SCE to AUX: Team Nissan is rejoicing. The Titan looks a lot better now, and their web site gives the beat-down to...

New Car Research

Get a Free Dealer Quote

Who We Are

  • Adam Tonge
  • Bozi Tatarevic
  • Corey Lewis
  • Mark Baruth
  • Ronnie Schreiber