By on October 27, 2009

Sometimes shooting par is good enough

Though we don’t have a [sub] for Consumer Reports‘ members-only data, their latest reliability survey summary has enough interesting tidbits to warrant a mention. Based on their subscriber base’s 1.4m autos, and using only data available for at least 100 examples of a given model, the survey is one of the better indicators of reliability out there (although when it comes to this topic there is no gospel). If nothing else, it’s hard to argue that CR’s reliability results aren’t influential, so sales are definitely at stake. The results? All Toyota/Lexus/Scion received ratings of “average” or better, an improvement over last year when CR found Camry V6, Tundra V8 4WD, and the Lexus GS AWD to be lacking. Honda/Acura and Subaru also showed extremely well where complete data was available, and Hyundai/Kia models were average or better except for Sedona and Entourage. Hybrids also scored surprisingly well, with nine gas-electrics scoring above average. But CR is making the biggest fuss over Ford, which they say is “on par” with the Japanese firms on all but a few truck-based models.  The rest of the Detroit firms? Not quite so much.

General Motors is a mixed bag. Among the bright spots is the redesigned Chevrolet Malibu; in its first year, the four-cylinder version is better than average and the V6 is average. The Buick Lucerne with a V8 and the Pontiac G6 with a four-cylinder are above average, and the Chevrolet Avalanche has improved to average.

But a quarter of GM models are still well below average in reliability. Some that didn’t fare well are fairly new designs that did well in our testing, such as the Cadillac CTS and the Buick Enclave, GMC Acadia, and Saturn Outlook SUV triplets. Chrysler trails the pack. Almost two-thirds of its products rate below average for reliability. The redesigned 2008 Chrysler Town & Country and Dodge Grand Caravan minivans earned low scores, as did the Chrysler Sebring V6 and Dodge Avenger sedans and the Jeep Liberty SUV. The Sebring Convertible has the worst score: 283 percent worse than average. The only above-average models are the Dodge Caliber hatchback and Jeep Patriot SUV.


Get the latest TTAC e-Newsletter!

168 Comments on “Consumer Reports’ Reliability Results...”


  • avatar
    rnc

    Ford has secured its position as the only Detroit automaker with world-class reliability. About 90 percent (46 of 51) of Ford, Mercury, and Lincoln products were found to have average or better reliability, according to Consumer Reports’ 2009 Annual Car Reliability Survey. The results were announced today at an Automotive Press Association luncheon in Detroit.

    Ford’s sustained production of vehicles that are as dependable-or better than-some of the industry’s best dispels the notion that only Japanese manufacturers make reliable cars. Other than the Toyota Prius, the reliability of the 4-cylinder Fusion and Milan ranks higher than that of any other family sedan. Both of those Ford Motor Company products continue to beat the Honda Accord and Toyota Camry

    Is what they actually said about Ford

  • avatar
    SherbornSean

    I’m waiting for some genius at Forbes to write an article on the correlation of low quality and ownership by the Democrat-controlled US government.

    Karesh needs better PR.

  • avatar
    Porsche986

    Congrats to Ford. Now the trick is to keep it going!

  • avatar
    VanillaDude

    If you’ve driven a Ford lately, this isn’t a surprise. Sadly, so much damage has been done to Ford by anti-US auto buyers justifying their Japanese purchases. “Japanese quality” became a mindless mantra for an entire generation of baby boomers now graying behind the wheels of Camrys and Accords across the US.

  • avatar
    basho

    I think this news comes as no surprise to auto enthusiasts. Hopefully Ford dealers get some unexpected traffic from people that are surprised by the news. I think it’s going to be extremely hard to pull customers away from Toyota or Honda because they have no reason to leave. But there is plenty of market share out there that is not Toyota/Honda and I think Ford could see a lot of conquest in that remaining portion of the market (as could Hyundai).

  • avatar

    SherbornSean: you volunteering?

    No one I’ve tried has made headway against the media’s belief that CR and JD Power provide all of the car reliability information that anyone needs. They are not interested in reading any critiques, or providing any themselves.

    For those who aren’t familiar with the (still uncorrected) flaws in CR’s approach:

    http://www.truedelta.com/pieces/shortcomings.php

    http://www.truedelta.com/pieces/cr_survey.php

    http://www.truedelta.com/pieces/newdots.php

    http://www.truedelta.com/pieces/anomalies.php

    We have at least one new anomaly. This year CR is reporting that the AWD Lambdas are considerably more reliable than the FWD Lambdas. How can a vehicle with additional parts be more reliable?

  • avatar

    Oh, and the connection between low quality and government ownership (a process begun by Republicans) is clear. Low quality ==> company goes bankrupt ==> government bailout.

  • avatar
    zorakcd

    As they say at GM and Chrysler; It’s good enough for government work!

  • avatar
    buzz phillips

    Now, if they could only get their resale
    value up (like Honda and Toyota), just say’n!

  • avatar
    jmo

    Sadly, so much damage has been done to Ford by anti-US auto buyers justifying their Japanese purchases

    I think the damage was done by US auto makers building crappy, under engineered, unsafe, unreliable, inefficient cars.

    They may have cleaned up their act but why should I trust them? If I’m happy with my camcord why risk a change?

  • avatar
    grog

    You guys don’t subscribe to CR? Sheesh. Contact me and I can login and get whatever data you’d like, assuming they have more stuff on the website that’s not available to the non-subscribing public.

    Here’s a question: how many of these world class Fords are being made at UAW plants? Seriously, this is not a drive-by flame attempt but rather an effort to point out that such data might show that the UAW isn’t the “productivity-sapping union” it’s otherwise portrayed as on this web site. I mean if the UAW can contribute to making world-class Fords it begs the question why aren’t GM and Chrysler doing the same thing. If these world-class Fords are being built/assembled in non-UAW plants, that too would provide some interesting hard metrics in which to judge the effectiveness of a unionized labor force.

    My recent experience with an 09 Ford Focus SES (totally tricked out) wasn’t great. Twas a rental (drove it for a week, highway and city) with 12K miles on it, leather interior, 5 speed manual, high-end sound system, whatever 4 cyl comes with that model. Gutless vehicle, ponderous handling, one plastic form behind the back seat window on the outside flew off on the interstate and the driver’s side sun visor fell out of fit’s socket. When compared side-by-side with my new Mazda 3, there was no comparison, the Mazda won hands down.

    Apparently that Focus wasn’t counted in CR’s survey.

  • avatar
    salhany

    I’m just shocked enough people bought Sebrings to report on their reliability to CR. CR gave them the lowest score by far in their sedans category just based on the road test, never mind reliability.

    CR’s been saying for several years that Ford’s reliability is quite good. The Fusion/Milan/MKZ cars in particular have been rated highly, in their FWD guises at least.

  • avatar
    dean

    In the time it took me to log in, jmo beat me to the punch.

  • avatar
    geeber

    VanillaDude: Sadly, so much damage has been done to Ford by anti-US auto buyers justifying their Japanese purchases.

    The damage was done by Ford itself, after years of installing crappy 3.8 V-6s and faulty transmissions in Windstars, Tauruses and Sables, and then ignoring the passenger car market to concentrate on trucks and SUVs.

    And I write that as a fan of Ford who wants to see the company succeed.

    VanillaDude: “Japanese quality” became a mindless mantra for an entire generation of baby boomers now graying behind the wheels of Camrys and Accords across the US.

    Mindless mantra? More like an established fact, backed up by independent surveys.

    Our 2005 Focus SE sedan, with 97,000 miles has been reliable and offers surprisingly good ride and handling (Ford now needs to work on the interior quality and overall refinement).

    But our 2003 Accord EX with 123,000 miles on the odometer feels newer, rides quieter and has an overall more “solid” feel than my mother-in-law’s 2005 Malibu with 50,000 miles on the odometer.

  • avatar
    KixStart

    Karesh: “This year CR is reporting that the AWD Lambdas are considerably more reliable than the FWD Lambdas. How can a vehicle with additional parts be more reliable?”

    What if all the AWD vehicless are built in a batch with unusual supervision or other attention? Or on a different line? Or they’re all built on Wednesdays? Or there’s a FWD part that’s particularly trouble prone which is not a part of the AWD vehicle?

  • avatar
    jmo

    How can a vehicle with additional parts be more reliable?

    Why is the Prius so reliable? Seems like number of parts as a reliablity indicator is a pretty weak measure when it comes to real life autos.

  • avatar
    wsn

    # Michael Karesh :
    October 27th, 2009 at 3:00 pm

    How can a vehicle with additional parts be more reliable?

    Of course it can. The Prius is ranked No.1.

    As for FWD and AWD, if everything is done right, the FWD should be more reliable. But chances are there would be multiple design and manufacturing flaws. So, it’s entirely possible that the AWD is more reliable than the FWD version.

  • avatar

    KixStart: there are often unexplainable anomalies like this in their data. The reason, as far as I can tell, is that the survey is so badly worded that even their large sample sizes cannot compensate for the poor quality of the data that results.

    The problems that affect the Lambdas are generally not powertrain-related. Unless they’re counting transmission reflashes as “serious problems.” But those affect both FWD and AWD.

  • avatar
    salhany

    I’m surprised they had enough data to say the Jag XF has poor reliability. It’s not a shock as problems have been reported elsewhere, but how many did Jaguar really sell? And how many Jag owners report to CR?

  • avatar

    wsn:

    By “additional parts” I meant all of the same parts as the first vehicle, plus a few others (AWD hardware in this case).

    The problem is in the data, not the vehicles themselves.

  • avatar
    KixStart

    VanillaDude: ““Japanese quality” became a mindless mantra for an entire generation of baby boomers now graying behind the wheels of Camrys and Accords across the US.”

    I’m going gray, true enough. But my children are also driving Toyotas, based on my experience, and they’re happy with them, too. I remember my Dad’s experience with cars… the GMs were better than the Fords, so I used to buy GMs. I didn’t switch for reliability, I switched for other reasons.

    But I’m staying for the reliability.

    And, can we get past “Japanese” or “Asian” cars vs “American” cars? Now that it’s looking like Ford is really a top-quality automaker, it should be clear to everybody that it’s the manufacturer, not the country. It’s not the country of origin, it’s whether or not the company building the cars puts a priority on getting it right, first time and every time.

    If a company believes that customer satisfaction is a profit driver, they’ll act accordingly.

    If a company believes that they can buy share with low prices, leverage massive fleet sales and cost-cut their way to happiness, they’ll act accordingly.

  • avatar
    jmo

    By “additional parts” I meant all of the same parts as the first vehicle, plus a few others (AWD hardware in this case).

    Any number of reasons. It could be that the vehicle has an issue with heat or humidity. If AWD is more likely to be sold where it’s cold for part of the year, then the AWD cars that are sold in New Hampsire would end up more reliable than those sold in Phoenix or South Florida.

  • avatar

    I tried to ask them (over on their blog) how long owners had had their highly ranked Insights when responding to the survey.

    That question didn’t make it through their screening process.

  • avatar
    Mr Carpenter

    I’m spending tens of thousands of dollars, when I am buying a new car.

    Why would I settle for “average” in anything?

    Especially reliability when I live in a climate severe enough in which one can die if you are stuck and broken down?

    Is anyone at GM or Chrysler liable to think that one through? Or the other sub-par automakers?

  • avatar
    Mr Carpenter

    When I buy a new car, I am spending tens of thousands of dollars.

    Why would I settle for average, especially in reliability?

    I live in a climate in which some of the year, you can die if the car leaves you stranded.

    Is anyone at GM or Chrysler or other sub-par car makers thinking this through?

    (Sorry for the repost but it didn’t come through the first time – unknown as to why)

  • avatar
    psarhjinian

    If a company believes that customer satisfaction is a profit driver, they’ll act accordingly.

    Ford made quite a to-do about improving quality and warranty claims four or so years ago. These are the dividends. Good for them, they finally clued in.

    It’s also nice to see people realizing that quality really is a holistic trait of vehicle design, rather than a function of origin, engineering theory or owner treatment. People have, for years, been trying to explain away the failings or trumpeting (if not exaggerating) their “wins”. Now that we have a domestic automaker that has put money into quality, it’s time to stop the “Made on Sunday/Buick owners baby their cars/CR is biased/Americans don’t understand how to care for a European car” excuse train that the others have been riding.

  • avatar
    George B

    I’m considering a 4 cylinder Ford Fusion as an economy car. Good fuel economy with room for 4 adults without being supersized. In my opinion, the Fusion is less ugly than the other cars in its market segment.

    # jmo :
    October 27th, 2009 at 3:08 pm

    Sadly, so much damage has been done to Ford by anti-US auto buyers justifying their Japanese purchases

    I think the damage was done by US auto makers building crappy, under engineered, unsafe, unreliable, inefficient cars.

    They may have cleaned up their act but why should I trust them? If I’m happy with my camcord why risk a change?

    I’ve owned both Ford/Mazda and Honda cars. I wouldn’t call the Ford/Mazda car crappy, but it suffered from premature failure of low bidder OEM parts. I’m willing to buy another Ford/Mazda car, but I would need to see a low initial price or a really long warantee to cover expected repair costs. The incentive to risk a Fusion over a Camary or Accord is better styling in a midsize package vs. ugly cars wrong sized to carry 4.5 adults.

  • avatar
    mountainman

    TrueDelta is the way to go for Reliability. Period.

    Now, from owning numerous cars over the years, I can also come to my own conclusions:

    84 Firebird – not reliable
    89 Z24 – not reliable
    93 Sentra SE-R – very reliable
    94 Shadow – not reliable
    97 Ford Escort – very reliable
    01 Silhouette – POS
    03 Sedona – somewhat reliable
    06 Fusion – not at all reliable and a POS to boot
    09 Impreza – very reliable

    Six domestics, and only 1 was reliable.

    Never again. Ever.

  • avatar
    Pch101

    I mean if the UAW can contribute to making world-class Fords it begs the question why aren’t GM and Chrysler doing the same thing.

    The UAW was likewise building Corollas, which generally show up in reliability surveys as excellent or above average.

    Blaming the workers has been a copout. Assembly quality is the result of production processes, which come from management. A well-managed assembly line is designed to have processes that can produce excellent products with mediocre workers; anybody with modest training should be able to contribute to a good result.

    An assembly line that relies upon craftsmanship and heroic individual workers is not truly an assembly line. Building a car is not like carving a table out of wood or painting a work of art. If an assembly line is truly that dependent upon worker quality, then its creator should be fired and the system reinvented, as the whole idea of an assembly line is to ensure that someone having a bad day can’t screw things up.

    Toyota reinvented the production process, and the quality skyrocketed because of their innovations. If Ford is figuring out how to use aspects of JIT and kaizen to produce better vehicles, then more power to them. Results beat whining every day of the week.

  • avatar
    carve

    I heard this before in the late 90’s. I bought a 98 Countour and found it to be a POS.

    The problem with the domestics are…
    1) They took their customers for granted and made decades worth of garbage
    2) When they realized it, they set their benchmarks on what the Japanese were currently making. They’d often not even hit those benchmarks, but even when they did the Japanese would already have something a generation or two better than what the Big3 were benchmarking.

    They’re never going to win all those lost customers back by being “just as good as”. They must be better. They must set the standard and let the Japanese play catch-up.

    That said, Ford does seem to be approaching this level. I’m still going to wait and see though. You know the old saying…Fool me once, shame on you, full me twice….can’t get fooled again.

  • avatar
    TomH

    As a US investor in two auto companies, it pains me that my tax dollars are propping up the bottom half of the CR list, and NONE of my investments made the top half.

    Best to worst brand list:

    1. Scion

    2. Honda

    3. Toyota

    4. Infiniti

    5. Acura

    6. Mitsubishi

    7. Lexus

    8. Hyundai

    9. Porsche

    10. Mercury

    11. Saab

    12.Subaru

    13. Suzuki

    14. Kia

    15. Mazda

    16. Ford

    — Middle —

    17. Nissan

    18. Volvo

    19. Buick

    20. Lincoln

    21. Volkswagen

    22. Pontiac

    23. Mercedes-Benz

    24. Audi

    25. Chevrolet

    26. BMW

    27. Mini

    28. GMC

    29. Saturn

    30. Jeep

    31. Dodge

    32. Cadillac

    33. Chrysler

  • avatar
    PartsUnknown

    You know the old saying…Fool me once, shame on you, full me twice….can’t get fooled again.

    That is hilarious. Great reference.

  • avatar
    Dynamic88

    If you’ve driven a Ford lately, this isn’t a surprise. Sadly, so much damage has been done to Ford by anti-US auto buyers justifying their Japanese purchases. “Japanese quality” became a mindless mantra for an entire generation of baby boomers now graying behind the wheels of Camrys and Accords across the US.

    It wasn’t mindless. It was based on real world observation and experience with cars.

    I drive a Ford daily (98 Ranger) and I’m not all that impressed, though not completely disappointed either. To be fair, they’ve probably improved since ’98, but then, to be fair, Honda and Toyota have probably not been standing still waiting for Ford to catch up.

  • avatar

    KixStart: And, can we get past “Japanese” or “Asian” cars vs “American” cars? Now that it’s looking like Ford is really a top-quality automaker, it should be clear to everybody that it’s the manufacturer, not the country. It’s not the country of origin, it’s whether or not the company building the cars puts a priority on getting it right, first time and every time.

    I second this. There are some very mediocre Japanese cars. I don’t hear anyone bragging about their Mitsubishi.

    I have relied on CR for all three of my car purchases. Unfortunately, their favorable assessment of Saturn (in the ’92 annual auto issue, I think) was premature, as–credit to them–they warned it might be.

  • avatar

    Pch101 +10

  • avatar
    John Horner

    @Michael Karesh

    Hmmm, I do have a CR subscription so I just looked up the GMC Acadia. The ’07 model scored much worse than average for the AWD version and simply worse than average for the FWD. For the ’08, both FWD and AWD scored worse than average.

    The ’08 Buick Enclave scored average for the FWD version and worse than average for the AWD version.

    I’m not seeing the AWD scoring better than FWD anomaly referred to in an earlier comment.

  • avatar
    jkross22

    What happened to Buick? I kept hearing from the defenders of GM how great and reliable and dependable Buicks had become. Or maybe that was just for the first 90 days/JD Power award.

    Oh well.

  • avatar
    Steven02

    @David Holzman
    I second this. There are some very mediocre Japanese cars. I don’t hear anyone bragging about their Mitsubishi.

    Mitsubishi is 6th on the list, ahead of Lexus.

    But, I also find it interesting how much of a different result CR can come out with when compared to other services.

    For 2009, JD Power had Buick and Jag tied for #1. Lexus was 3rd. Caddy was 9th. How can one explain these differences? Scion, number 1 here, was 31 on JD Power. Not that I agree one is more accurate than another, but both are respected and putting out very different results.

    The last comment I have is about the Lambdas and the Enclave only getting AWD as average. They say the Traverse was average. It has the same power train as the Enclave for both AWD and FWD. How does that work again?

  • avatar
    Juniper

    And except for Porsche the Europeans are all crap too. So either Cayanne or Camry it is.
    No Jag or LR? Are their volumes that low?

  • avatar
    Bimmer

    From TomH list Mercury is number 10 and Ford is 16. How could it be? It’s same vehicles put together by the same workers at the same factories! It just does not make any sense.

  • avatar
    cdotson

    Pch101 +1

    Glad to see someone finally tout the importance of more than just the goons who screw it together. Just because they screw it together doesn’t mean they’re they only ones who can screw it up.

    I’m far from a UAW defender but as an engineer I have to realize there’s a lot more to a quality product than capable assembly labor.

  • avatar
    CamaroKid

    CR statistical sampling and the results they publish are “suspect” at best.

    A Lucerne with a V8 (NorthStar LOL) has better then average reliability… NorthStars? Really? the engine that is known for headbolt that dissolve at 100,000 miles, burn a quart of oil every 1000 miles and blow up transmissions around 80,000 miles? That is better then average?

    The problem with a sample size of 100 is that the law of small numbers kick in and you have no idea what is going on and you just make up results…

    Lets say that in a sample of 100 you get 1 engine failure. Is that bad? Now lets say you get 2. Is that twice as bad? See the problem? Sure if you have a sample of 100 and 50 people have a transmission failure that it terrible… but you are looking for such SMALL statistical anomalies that all of the “data” is lost in sampling “noise”.

    On every CR “reliability” report there are two “different” models that are built in the same factory, by the same workers, on the same chassis, with the same power trains and one scores significantly better then the other…

    Sorry these reports are mostly “jack”

  • avatar
    skygreenleopard

    David Holzman:

    I second this. There are some very mediocre Japanese cars. I don’t hear anyone bragging about their Mitsubishi.

    While that’s a valid point, let’s strike a deal. You can make fun of Mitsubishi and Suzuki all you want (#6 and #13 on the list). They’re admittedly mediocre cars.

    I get to make fun of people who bought Buicks, Lincolns, Pontiacs, Chevies, GMCs, Saturns, Jeeps, Dodges, Cadillacs, and Chryslers (#19-#33).

    They I’ll resell my Integra for several thousand more than a same-year Taurus and laugh all the way to the bank.

  • avatar
    Verbal

    Bimmer: “From TomH list Mercury is number 10 and Ford is 16. How could it be? It’s same vehicles put together by the same workers at the same factories! It just does not make any sense.”

    The Ford lineup includes more trucks. Every Mercury has a Ford twin, but not every Ford has a Mercury twin.

  • avatar
    Loser

    Bimmer :
    October 27th, 2009 at 5:09 pm

    From TomH list Mercury is number 10 and Ford is 16. How could it be? It’s same vehicles put together by the same workers at the same factories! It just does not make any sense.

    Probably the same reason Buick is on top for GM, the old folks that buy these can’t hear the squeaks and rattles.

  • avatar
    ajla

    @Steven02:

    Keep in mind that the JD Power 2009 Dependability rantings dealt with 3-year old vehicles while the Consumer Reports survey (I believe) deals with the first full year of ownership.

    So what you see with the 2009 JD power ratings is actually how well 2006MY vehicles stood up.

    Then there’s differences in sample, survey questions, etc. between the two organizations.

    Like the intro says, there isn’t any one “auto reliability gospel” out there.

  • avatar
    Juniper

    skygreenleopard
    Why do you choose not to make fun of those driving European brands?
    Ah, Free pass for Otto!

  • avatar
    alex_rashev

    wsn,

    In regards to Prius reliability – it actually has LESS parts. A lot less. Your average 4-speed automatic transmission has a lot more intricate little parts and wear’n’tear items than two electric motors with controllers, a planetary gearset, and a battery. And don’t get me started on V6 engines, belt-driven accessories, 5/6/7-speed automatic gearboxes, and so on. Prius might not be the most handsome car, but the drivetrain is brilliantly simple, and thus inherently reliable.

  • avatar
    Pch101

    A Lucerne with a V8 (NorthStar LOL) has better then average reliability… NorthStars? Really? the engine that is known for headbolt that dissolve at 100,000 miles, burn a quart of oil every 1000 miles and blow up transmissions around 80,000 miles?

    My running theory is that this may be a function of the Buick demographic.

    Buick buyers are elderly – the average age is about 65. Older drivers drive quite a bit less than do younger drivers. Whereas the average American drives about 12,000-14,000 miles per year, depending upon the source of the study, the average elderly person drives about 3,000-6,000 miles per year, again depending upon the study.

    So an average elderly person buying a Lucerne will put perhaps 18,000-36,000 miles on it after six years (a timeframe included in Consumer Reports studies), and about half those amounts in three years (the timeframe for JD Power’s Vehicle Dependability Survey).

    You’re not going to see a lot of 100,000 mile failures among many members of the elderly driving pool; few of the cars will get used that much during the years that they end up in surveys. The Enclave has more problems because it has substantially younger buyers than do the other Buicks. If Buick is successful in getting a middle aged crowd to buy them, then expect the reliability to decline as higher mileage cars end up in the surveys.

  • avatar
    skygreenleopard

    ^ Good point. Seeing Bimmer so low is disappointing. I likely won’t be in the market for one for a while, though, but I also think they have a couple things going for them:
    1) the BMW Ultimate Service Plan, or whatever they call it, seems pretty comprehensive, though I don’t know the details of it.
    2) If you’re buying an X5, you’re already not getting a value buy, so I don’t know if these shoppers care as much about JD Power.

    It is disappointing I’ll never see a German car approach the top, though.

  • avatar
    Jerome10

    I’m so tired of hearing about “average”. EVERYONE should be familiar with this flaw of averages, yet everyone (including CR) puts so much weight on “above average” or “below average” and “below average” from CR means automatically cannot be recommended (if I’m recalling correctly).

    Lets assume their statistics are all completely correct, etc. Do I really give a crap if the “average” probability of failure of the engine is 0.002% and those crappy cars that are below average have a 0.003% chance of failure? You bet your ass I don’t care. Its the same really. Yet one is “below average”. Or if it is 0.001% and “above average” is it really better than the “below average” car? Nope.

    I wanna see the actual data. I wanna see some standard statistical stuff. If I remember back, that would include confidence levels, distributions, standard deviations, etc.

    This average crap numbs it down for dummies….maybe that’s what they want. Hell, if the cars were all actually very close, they wouldn’t really sell any magazines would they?

    This, more than anything else, bugs the crap outta me about CR. Nuts. Made even worse because so many car shoppers put so much weight on it.

  • avatar
    GS650G

    This is still more proof that chrysler as a car company probably should not exist. FIAT will come to realize what the Germans did eventually.

  • avatar
    cjdumm

    carve :
    October 27th, 2009 at 4:05 pm

    I heard this before in the late 90’s. I bought a 98 Countour and found it to be a POS.

    I’m sorry to hear there was another sucker out there who bought a ’98 Contour/Mystique and regretted it. Mine went through alternators as though they were hearing-aid batteries, and the interior fit and finish were horrible.

    It’s good to hear about an American success in building a decent car. If Ford can keep it up, they might win me back.

  • avatar
    johnthacker

    Blaming the workers has been a copout. Assembly quality is the result of production processes, which come from management. A well-managed assembly line is designed to have processes that can produce excellent products with mediocre workers; anybody with modest training should be able to contribute to a good result.

    Right, no UAW local has ever complained, frustrated, or intentionally sabotaged production processes suggested by management because those processes might result in workers being redundant, having to change their jobs, or just because management-labor negotiations or relations aren’t going well.

    Sure, “anybody with modest training should be able to contribute to a good result.” But there’s still a lot of room for a recalcitrant worker to screw things up. When management-labor relations are bad, there are tons of ways to work to rule or otherwise disrupt management processes.

    Next you’re going to tell me that in those cases, it’s solely management’s responsibility to discipline and fire those workers.

    And you’re going to tell me that if Toyota’s advanced production processes mean that the same job can be done by fewer workers and that Ford and GM would need to let some UAW workers go in order to copy them, then that would be no problem either.

    It’s much easier for Toyota starting a new plant, even UAW-staffed, in the US to start with then-current best practices. They don’t have any extra labor; they just make fewer initial hires.

    Of course the UAW contracts are indeed management’s fault as well, as management did sign them. (And even managers who weren’t there when the contracts were signed chose to take the jobs.)

  • avatar
    majo8

    While I’ll agree that Japanese cars have had better reliability than US cars, CR’s reliability ratings invite bias ( as Karesh points out in his link ), and are biased.

    Case in point: The Corolla and it’s twin, the Geo Prism ( both built by UAW workers ). In the 90’s, both these cars were built in the same factory, the same assembly line, and by the same workers, yet the Corolla received better ratings in CR. Look it up.

    The same can be said for the Ford Ranger and it’s Mazda twin — the Mazda got slightly better ratings.

    It may have something to do with the Japanese car owner not reporting that their car has a “serious” problem as much as the US owner would — if they had bought their car based mainly on it’s reliability and then it turned out not to be reliable, well……it would just make their purchase look foolish. And who likes to look foolish?

    Anyone who buys solely on the advice of CR is not getting the full picture.

  • avatar
    Steven02

    @Pch101:
    That only explains part of my comment. How did Caddy get so low here and so high in the JD Power Survey? How did Scion score tops here, and nearly the bottom in the JD Power survey?

  • avatar
    Pch101

    It’s much easier for Toyota starting a new plant, even UAW-staffed, in the US to start with then-current best practices. They don’t have any extra labor; they just make fewer initial hires.

    When NUMMI began, Toyota hired about 85% of the staff that GM had employed at the Fremont plant. This is despite the fact that Fremont was notorious for being one of GM’s worst plants, with one of the highest absentee rates of any of its factories.

    Mr. Rattner referred to the “quiet arrogance” of GM lifer Rick Wagoner. You can bet that this arrogance wasn’t so quiet with the plant managers.

    GM gets the output that it deserves. If you treat people like cattle, don’t be surprised if they produce to the standard that is provided. Labor is supposed to be managed.

  • avatar

    Pch101 +1

  • avatar
    Z71_Silvy

    This, from the same institution that, until recently, would blindly recommend Toyota?

    Who, on this planet, takes CR seriously anymore?

    They are a joke…..and have lost all credibility.

  • avatar
    Durask

    Agree with some posters above like Jerome.

    CR statistics are GARBAGE.
    Once again, CR statistics are GARBAGE.

    What is “above average” and “below average”? What is “average”? What’s the difference between “below average” and “above average”? Is the difference statistically significant? Does CR tell you whether the difference between say Honda and Dodge is statistically significant? If they do not, their stats are nothing but junk.

    A simple example of how statistics can be misinterpreted. Suppose studies show that people who drink diet soda have higher risk of getting lip cancer.

    Suppose if you don’t drink diet soda, you risk of getting lip cancer is 1 in a million and if you drink diet soda, your risk of lip cancer is 2 in a million. Now you can say “OMG people who drink diet soda are TWICE as likely to get lip cancer, I will never touch diet soda again” or you can say “1 in a million or 2 in a million is still nothing to worry about”.

  • avatar
    wsn

    Pch101 :
    October 27th, 2009 at 3:50 pm

    The UAW was likewise building Corollas, which generally show up in reliability surveys as excellent or above average.

    That’s a political agenda driven, one-off and unsustainable experiment. I will buy that, when all Toyota cars are built by UAW and still make money and get good CR scores.

  • avatar
    taxman100

    Mercury does so well because it still sells the Grand Marquis – the best platform Ford makes for durability and reliability. Check out the JD Power long term reliability studies – it is always among the best, regardless of price.

    I know – I’ve owned two, both with more than 100,000 miles on them, and other than some front suspension work around 100,000 miles, the rest of the car will last until you get tired of it and that old car stink high mileage old cars develop.

    When the Grand Marquis goes away, I’m done with Mercury.

  • avatar
    wsn

    Pch101 :
    October 27th, 2009 at 6:10 pm

    When NUMMI began, Toyota hired about 85% of the staff that GM had employed at the Fremont plant. This is despite the fact that Fremont was notorious for being one of GM’s worst plants, with one of the highest absentee rates of any of its factories.

    There is a difference between “UAW-involved plant” and a “UAW-controlled plant”.

    For Toyota, NUMMI is only a “UAW-involved plant.” Toyota can walk away from it with very little loss if needed. Therefore, the UAW workers have to be diligent. In other words, the UAW is the pig, while Toyota is only a chicken. (If you know my analogy.)

    However, for other GM plants, UAW literally controls the plants. The company would close doors when they go on strike. They are both pigs. A pig can threaten another pig, but not a chicken.

  • avatar

    CR refuses to release the actual repair frequencies. They say this is because numbers would confuse people.

    But I think it’s also because they know that their numbers are clearly inaccurate in absolute terms. The thinking is probably that as long as the ratings are relative the errors cancel out.

    This is one of the main reasons I started TrueDelta’s survey. We post the actual repair frequencies, to make the size of the differences clear (hence the name of the site).

    http://www.truedelta.com/car-reliability.php

  • avatar
    Happy_Endings

    CR statistics are GARBAGE.
    Once again, CR statistics are GARBAGE.

    ALL statistics are garbage if the user uses them as the gospel and not a guide. Derek Jeter is considered a vastly superior player than Mario Mendoza, yet if someone who doesn’t know baseball compared the two players career batting averages, they’ll find Jeter is getting exactly one more hit per ten at bats than the namesake of the Mendoza Line and someone who is often considered the worst hitter in the history of the game.

  • avatar
    Daniel J. Stern

    @Jerome10:

    +10

    (I feel all superior because I own a car that got rated “red dot with white spot” but my dumb neighbor bought one that only got rated “white dot with red perimeter”.)

  • avatar

    Okay, I found the details on the Lambdas.

    The FWD model is 21% below average.

    The AWD model is 19% below average.

    So they’re essentially the same. But CR draws an arbitrary line at 20% below average. So one is “average” while the other is “worse than average.”

    How many people will opt for AWD as a result?

  • avatar
    fincar1

    I subscribed to CR back in the day…late 70’s iirc. I wrote to them and asked them how many cars had to be in the survey before they put out statistics on them rather than saying “Insufficent data”. My main reason for asking was that they had data on Corvettes, and I couldn’t see that many Vette owners being CR subscribers. No answer, of course. I let the subscription go when it expired.

  • avatar
    Telegraph Road

    @Michael Karesh
    Do you have any insight as to why JD Power Dependability (_not_ JDP Initial Quality!) and CR Reliability rankings differ widely and consistently (i.e. year after year) on many brands. CR loves Scion and berates Cadillac–JDP just the opposite. Is it the age of the survey respondent (the young check more boxes)? Please give us your insight. Thanks.

  • avatar
    fincar1

    While I do not think very much of CR’s statistics (I posted on that but don’t know if it made it through or not…got a “couldn’t find server” error message) I do believe that their excuse that the actual numbers would simply confuse people is correct in a lot of cases. The reason for this is (1) mathematical education in this country is generally pretty poor, and (2) a lot of people can’t even make change correctly without the cash register being programmed to figure it out.

    If CR’s statistics actually make some sense, it would help their credibility to post them on their website so those who wanted to see them could.

  • avatar
    Bridge2farr

    I’m truly surprised CR would give Ford the rating they got simply because CR has an anti-American manufacturer bias. Sorry, I said it and many of you know it is true. I wouldn’t put an ounce of credibility in their report.

  • avatar
    Telegraph Road

    @Michael Karesh
    I just read your links above. Good work. Do you have any similar links for JDP?

    BTW, as a professional statistician, I fully concur with your firm’s support of prospective rather than retrospective analysis, despite the smaller sample sizes.

  • avatar
    KixStart

    VanillaDude: “Sadly, so much damage has been done to Ford by anti-US auto buyers justifying their Japanese purchases.”

    I get so tired of hearing things like this. All the damage that was done to Ford in our household was done by Ford. The vehicle sucked and Ford bent over backwards to make sure we knew they couldn’t care less.

    If Ford had made us happy, we’d still be buying Fords. Everybody’s like that.

  • avatar
    Lumbergh21

    I agree that without knowing what those dots really mean, they are meaningless (seems obvious, doesn’t it?). Along with providing repair frequencies, it would also be nice to include average cost of repairs and number of days in teh shop. I care about three things when it comes to reliability: 1) how often does it break down, 2) how long will I be without my car, and 3) how much is it going to cost me? Without all three pieces of information, I don’t have a true picture of what this car will potentially cost me in time and money. A car with an average number of quick and cheap repairs is greatly preferrable to one that suffers half as often from head gasket failures or even worse clutch failures in terms of both time and money. Would it really be that hard to include the cost of repair and time in the shop information?

  • avatar
    Lumbergh21

    fincar:

    +1

  • avatar

    Telegraph Road:

    CR’s and JD Power’s results will differ because they ask different questions, and CR’s question has a major flaw that opens the door wide for any respondent biases.

    You might be the first person to understand why I use a prospective longitudinal survey!

    Okay, I’m sure others do. But they tend to keep quiet on public forums–probably a sign of intelligence.

    Amateur statisticians like to tell me my results are crap because I don’t use random sampling. But the prospective approach largely compensates, especially when you consider that even those who use a random survey have response rates in the low 20s.

    Always trying to fix the smaller sample sizes.

    I don’t have as many links for JD Power, because their flaws are less severe.

    http://www.truedelta.com/pieces/jdpower.php

    http://www.truedelta.com/pieces/IQS2006.php

    The real question: why are critiques like these and those on CR entirely absent from the media?

  • avatar
    Telegraph Road

    @Michael Karesh
    I just read your links above. Good work. Do you have any similar links for JDP?

    BTW, as a professional statistician, I fully concur with firm’s support of prospective rather than retrospective analysis, despite the smaller sample sizes.

  • avatar

    Lumbergh21:

    I’d like to provide days in the shop and costs. But both would require larger sample sizes than my current metric.

    I will be introducing two new stats next month: Nada-odds and Lemon-odds. The first: your chances of having no repairs over the course of a year. The latter: your chances of having three or more repair trips.

    With more participants, I could do a lot more. So, how to get more participants?

  • avatar
    Telegraph Road

    @Michael Karesh
    Indeed. Low survey response is an indicator of selection bias as you must know. The angry and the young respond–we old, happy folks just hang up.

    So why is it that Scion aces CR, but fails JDP? Yet Caddy does the opposite?

  • avatar
    srclontz

    As a subscriber, I took a look and noticed a few problems. The Toyota Matrix and Pontiac Vibe have reliability data for the 2.4 motor going back to 2003. Pretty incredible that they have data for a car that wasn’t even produced. It also seems odd that they split the Matrix/Vibe apart by motor, however the competing non-turbo Impreza is separated by trim package (which the Outback Sport package largely is) but the Forester is separated by transmission. Such differences between how different models of cars are compared makes it more difficult to rely on the data they produce.

  • avatar
    car_czar

    Jerome10: +1

  • avatar
    KixStart

    srclontz,

    That motor has been around since 2001 or so, so it has some history. It’s the same one used in the Camry.

    Bridge2farr: “I’m truly surprised CR would give Ford the rating they got simply because CR has an anti-American manufacturer bias. Sorry, I said it and many of you know it is true. I wouldn’t put an ounce of credibility in their report.”

    That makes no sense.

  • avatar
    Telegraph Road

    @Michael Karesh
    “Why are critiques like these and those on CR entirely absent from the media?”

    Unfortunately the media, with few exceptions, are primarily concerned with conventional wisdom. CR is conventional wisdom. JDP is Detroit’s fee-paid lackey. You are–who are you again? ;) You must know you’re fighting an uphill battle.

  • avatar

    Telagraph Road: Oh, I know. Didn’t realize it initially. Silly me, I thought journalists sought to inform the public. Now I know better. Whatever gets the eyeballs, with a minimal amount of kickback.

  • avatar

    srclontz: I don’t see any reliability data for the Matrix and Vibe that is split by engine. Are you sure you didn’t misread something?

    The only results I can find do not specify any engine. They do say the Matrix is 20% better than average, and that the Vibe is 35% better than average. So much for people who claim that the Pontiac badge hurts reliability ratings in CR.

    I’ve written a number of critiques of CR, linked above. In none of them do I accuse CR of being biased against the domestics.

  • avatar
    PeteMoran

    My employer used to track the $$$ of warranty provision made in quarterly accounts as future liability compared to unit sales and sales value. (With various corrections for model counts, likely model unique parts etc).

    Not sure why we stopped, but IIRC the domestic trend was very very very slowly closing on the transplants after a huge gap previously.

    The last time I noticed it, GM had reduced their provision, but maybe it tracked exactly with sales.

    I always thought it was an interesting metric.

  • avatar

    The domestics’ warranty costs have been continously decreasing. It would be helpful to have those figures.

  • avatar
    srclontz

    KixStart:
    While the repair history may be similar for the 2.4 motor in the Camry, the 2.4 was not offered in the Matrix until much later than 2003.

    Michael Karesh:
    Perhaps I am doing it wrong, but I clicked on the link in the above article titled “Research reliability ratings for 2009 Models by Category” then selected every car type, clicked “Get Results” then clicked on the Reliability tab. I checked again, and it has separate entries for the 2.4 & 1.8 Matrix and Vibe. By the way, your site is great, I fill out my survey monthly and encourage everyone I know to sign up.

  • avatar
    dejal

    VanillaDude: “Sadly, so much damage has been done to Ford by anti-US auto buyers justifying their Japanese purchases.”

    I find your “anti-US” remark offensive.

    3 Accords over 28 years. 609,000 miles. Current car is a 98 4 door with 234,000. Original clutch. Original exhaust. Air still blows cold. The only electrical issues are that the remote trunk open using the key fob no longer works, and the HVAC section of the dash has a couple of bulbs out.

    It’s not a Not a highway commuter. I’ve counted the shifts in the past back and forth to work and I shift about 200 times in 52 miles (lots of stop and go). The lip of the drivers side rear wheel well is rusting away (I live in New England). Have I sunk money into them? Yup. Timing belts every 100,000 or so. Bushings wear out. Starters break (around 220,000 miles). Engine mounts fall apart. Caliper replacements. Big deal. The biggest disappointment is brake rotors. They warp easy whether OEM or aftermarket.

    Speaking of clutches, I’ve had only 1 clutch replacement among the 3 cars.

    So, I’m delusional about clutch longevity.
    I’m delusional about the original exhaust.
    I’m delusional about 609,000 miles.

    I guess I’m supposed to take one for the team.

    That being said, my sister was looking to replace a 92 Taurus that our late father owned and she took over. I tried to push her to used Sonatas + Fusions as I believe they are almost as good and at a better price than a used Honda of the same vintage. She had recently rented a low mileage Impala with leather and power everything and said it was crap in the 500 miles that she drove it.

    She’s had Hondas, Subarus, Fords, Volvos and a Suburban in the past with pretty good luck. She actually would have been very happy with the previous model Taurus if she could have found some on dealers lots. She ended up with a 2006 Accord VP (Very Plain, Value Package). Nice car.

    She knows Honda is capable of very high mileage and is pretty reliable. It’s her money, how is that being a anti-US auto buyer?

    I’ve had a great experience with a Japanese brand and I’m anti-US?

    My sister has had Japanese, American + European and decides to buy Japanese and she’s anti-US?

    I’m guessing that this is your favorite song:

  • avatar
    coast

    I dunno. We went to a Ford dealership last weekend to look at the Fusion Hybrid. Test drove one with 23 miles on the odo and an MSRP of $33,210. After the drive, I opened the trunk. There, on the left and right sides of the inside of the trunk lid, were two corrugated plastic conduits (for wiring I suppose) that were ill-fitted to the trunk liner and, at the point where they emerged from the liner, sloppily wrapped in–wait for it–electrician’s tape, with the raggedy ends of the tape already coming loose. This was in full view at eye level.

    My wife is pushing for a Prius. Guess how our conversation went after I closed the trunk?

  • avatar
    Telegraph Road

    Please, if you’ve had a bad experience with an automobile brand, I can fully understand why you wouldn’t want to buy another (as payback). But please don’t generalize from your sample of 1 (or 2 or 3 or 4), and please spare us the details. Leave the generalizations to CR, JDP, and True Delta–they work with much large sample sizes, and more rigorous methods.

  • avatar

    srclontz: I reproduced that page. They’re simply showing the same reliability dots for both engines. They don’t break down results by engine for these models.

    I don’t either, not only because my sample size is too small to do so, but because these cars require virtually no engine repairs in their first couple of years. Few cars do these days.

    I’m glad you like the site. Unless you have a repair to report, though, it should only be necessary to respond to the reliability survey once each quarter, not monthly.

  • avatar

    Found a drivetrain anomaly that’s not as easily explained as the Lambdas:

    CTS V6 RWD: 57% much worse than average

    CTS V6 AWD: 41% worse than average

    16% is a large difference in terms of CR ratings–it gets you from a half-black to a full-black dot.

    Looking at the various dots, the big differences between these cars appear to be in the areas of body hardware and brakes–not the drivetrain.

  • avatar
    kipling

    Karesh, et al., here is a “sort of source” for warranty claims. The data is in the SEC 10-Q and 10-K reports if you want to extract it yourself.

    Please don’t draw unfounded conclusions from this. These values are what each company sets aside for warranty costs. It doesn’t give number of claims, only the cost of the claims for the overall set of cars under warranty per quarter.

    Japanese Costs:
    http://www.warrantyweek.com/archive/ww20090702.html

    vs. Ford and GM:
    http://www.warrantyweek.com/archive/ww20090723.html

  • avatar
    Maxb49

    As a statistician, I am troubled by Consumer Reports methodology. If I recall correctly, Consumer Reports relies on their readers to send in their car data. Whichever way you look at it, this constitutes a HUGE selection bias.

  • avatar
    tedward

    My problem with CR is that I want more inside baseball details. If a car has a single issue dragging down the relative score in a category, I want to know if it’s something expensive, exactly what it is, and if a fix is available anywhere. Hell, I want to know who’s fault it was exactly. This is the kind of detail I get from fellow car enthusiasts when I ask about their cars, not just a, “oh, above average major engine repair wise.” Why not be more informative? They could give the information without numbers if they insisted, that would be enough. They could also temper the significance of their statistical data in this way (the responsible thing to do), and without losing credibility in the process.

    The thing that makes CR’s faults annoying is the manner in which it is used by average shoppers. They actually stop at the overal reliability score. If you look in the back you can, on older cars certianly, just realize that a drivetrain low score has a bigger cost potential than an interior failure, common sense makes it a little useful. The problem is, newer cars have an ability to be expensive in unlikely places, and the format doesn’t address this.

  • avatar

    Telegraph Road: I can’t count how many times I’ve had people tell me that a minimum sample size of 25 is way too small (and it is certainly cutting it close–I’d certainly prefer 100+), then tell me that they know CR is correct because the four cars they’ve owned have confirmed CR’s vague ratings.

    With next month’s Nada-odds stat, I’m finding that even with the least reliable cars you have about a one-third chance of making it through a year with no repairs. So having one car that requires no repairs proves nothing–it’s within the realm of possibility with nearly any car.

  • avatar
    Telegraph Road

    @Maxb49
    Indeed. The angry and the young send in responses. The rest of us curse our junk mail.

  • avatar
    PeteMoran

    @ kipling

    Great get. I think that explains why our company doesn’t get involved in counting this stuff.

    There is a journal/data-source for your specific industry. From Rooster Breeder Weekly to Warranty Week or NAMBLA, they’re all out there.

  • avatar
    Telegraph Road

    @Michael Karesh
    The important issue is that with sound methods, standard errors can be trusted. With confounding selection bias, they can’t.

  • avatar
    Telegraph Road

    @Maxmb49, Michael Karesh:
    Statisticians have a term for what CR and JDP do: the Self-selected Listener Opinion Poll–or “SLOP” for short. I recall a few years back when the National Academy of Sciences surveyed their members using SLOP. The statisticians revolted and the NAS apologized.

    Nevertheless, it beats a sample of 1 (or 2 or 3 or 4).

  • avatar
    coast

    Telegraph Road: I understand your desire for rigorous methods–in fact, as a professional journalist who is not beholden to any advertisers, I share it. At the same time, I know that the details of a product–even *a* detail–may reveal something about the company that produced it.

    Does a manufacturer’s visible, sloppy fix with electrical tape indicate that a new car is unworthy of buying? Maybe not. But which probability is greater: That it reveals something about the standards set by the people who run the company…or, as you imply, that it is a case of an employee autonomously deviating from standards set by the company? Which answer does not challenge your confidence in the quality of the product?

  • avatar
    Telegraph Road

    @kipling-thanks! good stuff.

  • avatar
    jmo

    What does everyone think about legislation to require automakers to publicly release their warranty data? How many claims were made, what they were for, how many of the claims were paid and how many were refused, etc.

    It seems like it would be a huge boon to consumers.

  • avatar
    Telegraph Road

    And while I’m ranting… does anyone at CR or JDP know what a standard error is? Or is that for internal consumption only? They hide their methods and statistical confidence for one reason and one reason only–to preclude external scrutiny. Yet the media lap it up. I’d like to see them get their results published in something other than a newspaper.

  • avatar
    mtypex

    I concur that CR data doesn’t include enough detail. However, if a car is an obvious bland chore to drive, they will note so. It’s not the best, nor is it the worst.

    The low placement of BMW is scary. It gives me pause (and a TSX is $10K less than the 3-Series I want). Until then, nobody here should be reselling ANY Integras, especially under 200K mileage on the odometer.

    I went through course after course in grad school of interpreting standard errors and controlling for selection bias. The techniques were all so glamorous-yet-obscure that I have not found any means in the real world to apply them. If I remember correctly, to correct for the selection bias, you’re basically randomly selecting other data points, that is imputing them based on some hackneyed statistical theory that no honest statistician would ever advocate under oath of law.

    The standard error of Chrysler’s error is big enough to drive a Jeep Commander through, even with a Dodge Caliber on the rough. I’ll stop there!

    Well, one more quick note, on Fords: am I gonna trust outdated Mazda and Volvo engineering, or am I gonna get a Honda? ok, answered the question. Hands down. (Noooo – gotta have a big chrome grille!!!)

  • avatar
    Daniel J. Stern

    @Telegraph Road:
    Nevertheless, it beats a sample of 1 (or 2 or 3 or 4).

    Oh, I donno about that. It depends on the goal. If a straight-line fit is required, your best bet is to obtain only two data points!

  • avatar
    Geotpf

    There are groups of manufacturers in terms of overall reliability. Basically, there’s three groups, as far as I can tell.

    Group one: Honda/Acura/Toyota/Lexus/Scion/Subaru

    Group two: Ford/Lincoln/Mercury/Hyundai/Kia/all other Japanese players

    Group three: All other Detroit players/all European players

    Ford/Lincoln/Mercury, Hyundai/Kia, and Nissan/Infiniti used to be in group three with the rest of the losers, but have improved recently. Nissan/Infiniti was only in group three for a couple years due to horrible teething problems in their newer Mississippi plant.

    Moral of the story is to avoid group three, and be very wary of group two. Now, individual models in the lower two groups may be reliabile, but taken as a whole, they aren’t. But all, or almost all, models from group one should be reliabile.

  • avatar
    Maxb49

    Indeed. The angry and the young send in responses. The rest of us curse our junk mail.

    I’d like to see the true descriptive statistics of Consumer Reports readers. It’s just a guess, but an educated guess that a Consumer Reports subscriber is highly likely to own an Japanese Car. Credible statistical studies have shown that Japanese car owners who have mechanical problems with their car are unlikely to actually consider them problems, and report the car as being trouble free. There is likely enormous bias in Consumer Reports data.

    I’m openly anti-Japanese cars. Not only because I support American industry. I’ve never had good luck with the Japanese cars I owned. Models recommended by Consumer Reports.

    o.s. Does anyone remember the baby seat scandal?

  • avatar
    brandloyalty

    I may be mistaken, but I recall that the AWD Aerostar van was much better quality than the RWD version, which was terrible. It could be that AWD versions are top end of the model range, and may well be built better than the cheaper versions. Or that people who can afford the AWD versions take better care of them.

    While I doubt most of the people on the production lines can tell if a given car’s going to be 2WD or AWD (and so perhaps build the AWD’s better), the drivetrains may or may not be substantially different between the two versions. AWD, though more complex, has the potential to otherwise reduce strains on the rolling gear.

    Since buying a Suzuki I’ve been curious about CR and Suzuki. As most people know, CR and Suzuki had a major court dust-up about capsizing Samurai’s. I find it odd that the Grand Vitara does significantly better on Karesh’s truedelta than in CR’s surveys. Suzuki, now the world’s 9th largest car manufacturer, and for whom North America is a small market, doesn’t seem to have image problems outside North America.

    Seems to me CR did a reliability comparison of the 2006 cuv’s. They included the 2006 Rav4 even though it hadn’t been on the market long enough to meet their own survey’s rules for inclusion. Despite being an all-new generation with no data, they included it AND gave it a top rating because the previous generation had done well.

    Most cars these days have excellent reliability. I’d like to see stats on the quality of the support the dealerships give when an owner comes in with a problem. I’d rather have three problems sorted out fairly and quickly than two problems they are determined to weasle out of.

  • avatar
    PeteMoran

    Credible statistical studies have shown that Japanese car owners who have mechanical problems with their car are unlikely to actually consider them problems, and report the car as being trouble free.

    Actually, I think the argument is the opposite. Our marketing guys often speak of it;

    Complaint frequency is directly related to expectation.

    As an example; Lexus owners have a high expectation of fault-free ownership, and are extremely likely to report even the slightest issue. Lexus themselves test this regularly via contact with their owners, further raising the expectation and encouraging reportage.

    BMW tried to emulate Lexus (encouraging feedback) and got a nasty shock. I would suggest they’ve lifted their game somewhat since (late 1990s).

  • avatar
    skor

    I believe that there is some truth to the accusation that people value Japanese cars over American cars because the Japanese cars are presumed to be better/more reliable.

    Back in the day, the Ford Probe and Mazda MX6 were produced on the same assembly line at Mazda’s Flat Rock, Michigan plant. The cars were mechanical twins, but the Mazda sold for considerably more money on the used market than did the Ford — taking into account mileage and condition.

    The same was true for Mitsubishi’s that were branded as Chrysler’s. The Mitsubishi badged cars sold for more money.

  • avatar
    DweezilSFV

    Mr. Karesh: your observations about C/R are ignored by the establishment media because it’s so much easier to not question them. They are also part of the establishment press which long ago forgot about who, what, when, where, and why in writing a story.It’s journalism by press release.

    It’s the same reason they get all google eyed at any test the IIHS does. Headlines over substance. They’re an institute so therefore an authority, therefore their conclusions must be truth.

    Taxman100: that “old car stink” is one of my fovorites. It’s also the smell of money accruing interest in the bank. New cars smell like debt after the initial offgassing of all the plastics dissipates.

  • avatar
    Pch101

    Consumer Reports relies on their readers to send in their car data. Whichever way you look at it, this constitutes a HUGE selection bias.

    Sorry, but no. Unless you believe that CR subscribers who buy domestics have uniquely bad luck with their domestic cars (perhaps the salespeople make a point of selling their lemons to them) or you believe that CR readers are more inclined than the average person to lie about the reliability of their vehicles, that doesn’t make much sense.

    The CR survey is quite good because the number of respondents is large (hundreds of thousands). Unless you want to believe that there is some vast Asian conspiracy within that pool, the statistical noise of their respondent pool should be reduced substantially by the sample size.

    My problem with CR is that I want more inside baseball details.

    JD Power provides those to those who pay for their data.

    What you need to remember is that the accuracy of a survey will come in part from the method of collection. One of the best aspects of the CR survey is the fact that it is simple. There are very few questions, and they aren’t difficult to fathom, which leads to the answers being more complete. Demanding details of those surveyed will lead more people to rush through it — many don’t wish to commit much time to it — and to omit quite a bit.

    In any case, the surveys generate correlate with each other and with reality. The domestic car fans are simply pissed off that the facts don’t support their Lutzian “perception gap” nonsense.

    I’m tired of the whining. Every time I hear some Detroit fan boy whine about the American consumer, it makes me determined to avoid domestic purchases until the proof of their alleged equality is absolutely proven beyond all doubt, with no room for error.

    The entitlement mentality tells me that the domestic boosters are not to be trusted, and they will keep selling the dream just for the sake of selling, even if it is ultimately a lie. No thanks, I’ll export my cash if I have to (and that may be to such exotic destinations as Kentucky and Ohio, where many of our “foreign” cars are built.)

  • avatar
    Pch101

    For Toyota, NUMMI is only a “UAW-involved plant.”

    That’s a crock. Toyota had a contract with the UAW, to the point that Toyota hired most of the same workers who had worked for GM. Toyota got better results with a workforce that was allegedly one of GM’s worst.

    The reality is that Toyota does a better job of operating assembly lines. The team-building methods used by Toyota and Honda, which emphasize unit quality instead of volume throughput, result in better cars.

    Enough of the politically-motivated right-wing gyrations that put anti-union ideology above business reality. It’s quite simple — the best Japanese car makers don’t use the same production methods that have been traditionally used by the domestics, and those differences contribute to better assembly quality. These differences are due to management, management and management.

    We should stop bashing on the little guy, who has very little to do with much of what occurs, and focus on who actually operates the business. Give some credit to companies such as Toyota for doing a better job of managing their businesses.

    These companies are **not** all the same, and they should be judged based upon the qualitative value of their differences. If the domestics want to improve, turning workers into allies instead of opponents would do them some good, but there’s that little “quiet arrogance” problem that gets in the way.

  • avatar
    don1967

    I seriously doubt that adding All-Wheel-Drive or a Hybrid powertrain makes any car more reliable. It likely just makes the owner more enthusiastic when filling out surveys.

    Hybrid cars in particular enjoy a quasi-religious following. When you’re a member of the Church of Global Warming, you just don’t go around badmouthing St. Prius of Toyota.

  • avatar
    Bridge2farr

    I believe it is true that a typical CR subscriber is “friendly” towards the Japanese product. They conveniently omit talk of problems with their Honda by calling it “a great service experience” as opposed to a vehicle defect. They generally speak against domestic product as a way to justify to themselves and their circle of friends their purchase decision is valid. I witnessed a consumer telling me how happy they were with their Honda experience. The dealer helped them tremendously during their lemon law Accord replacement! I found that both sad and hilarious.

  • avatar
    geeber

    Maxb49: Credible statistical studies have shown that Japanese car owners who have mechanical problems with their car are unlikely to actually consider them problems, and report the car as being trouble free. There is likely enormous bias in Consumer Reports data.

    Can’t buy it. One of my acquaintances is an employee of Pennsylvania’s largest lemon law firm. He investigates claims by plaintiffs and works with manufacturers to settle claims before they go to court.

    He said that Honda and Toyota have the best reliability by far, and are more likely to settle to avoid a lawsuit.

    The Germans – particularly Daimler and VW – have terrible reliability and are extremely arrogant and unwilling to negotiate.

    The domestics are in the middle.

    He specifically said that Honda/Acura and Toyota/Lexus customers are more likely to go ballistic over even minor problems, because they expect their cars to be perfect. Some of the problems aren’t even mechanical in nature – people want to use the lemon laws to force Honda to buy back their vehicle because they find that the seats are uncomfortable, for example.

    Ignoring problems is far more likely among domestic car owners, in my experience. I’ve had relatives and friends experience major problems with a domestic car…and turn around and buy another of the same brand.

    When someone tells me that he or she works as a mechanic, I make it a point to ask which manufacturer makes the best vehicles (from a reliability standpoint). The answer is always the same – Toyota and Honda. They do agree, however, that GM and Ford – particularly Ford – have improved dramatically over the past 4-5 years.

  • avatar
    geeber

    Pch101: It’s quite simple — the best Japanese car makers don’t use the same production methods that have been traditionally used by the domestics, and those differences contribute to better assembly quality.

    Not to disagree, but I thought that they also work more closely with suppliers to make sure that parts are built within specifications and set tougher performance standards during prototype testing, too.

    The parts being assembled by workers are therefore easier to install (which reduces the chance for error) and much more likely to perform as the customer expects.

    I know that from reading posts on other websites from Ford insiders, the company has been very aggressive in borrowing quality control techniques from Mazda and Toyota, and it looks as though the effort is paying dividends. What worries me is that I still hear stories of Ford driving the suppliers too hard.

  • avatar
    Pch101

    I thought that they also work more closely with suppliers to make sure that parts are built within specifications and set tougher performance standards during prototype testing, too.

    The parts being assembled by workers are therefore easier to install (which reduces the chance for error) and much more likely to perform as the customer expects.

    That’s also certainly true, and describes just another aspect of what makes some car companies better than others.

    Detroit can’t improve until their leadership acknowledges that they have been inferior and improvements therefore must be made. At least Ford ended up with an outsider, and Chrysler may even have a shot because it now has foreign managers who don’t feel the need to buy in to the local status quo.

    I have less hope now for GM, which did not get the benefits of new management, and seems likely to rehash the same mistakes that it has made for decades. It appears that the feds gave up, and just felt thankful that the crash landing will occur at a time when the economy can better handle it.

  • avatar
    NickR

    “Japanese quality” became a mindless mantra for an entire generation of baby boomers now graying behind the wheels of Camrys and Accords across the US.

    No, not mindless. Our family’s most recent Fod experience was a mid-90s Crown Vic. In the end, it ran for a long time. However, the teething troubles at the beginning soured us on the car forever. The wiper system took three major overhauls to work properly (how complex is a wiper system?), the a/c failed, the paint peeled off (which Ford refused to do anything about), it developed a mysterious water leak and the oil pan rusted out prematurely. Even when the mechanical problems were straightened out (at great expense), we were still stuck with a car looked like shit. It was am embarassment to be seen in. In the end it was a reliable winter beater but that’s the best that can be said about.

    More recently, my neighbour’s experience with his Ford Focus was even more discouraging. A lemon of epic proportions.

    Some of Ford’s cars appeal to me, but I am hard pressed to cut them any slack.

  • avatar
    geeber

    Pch101: I have less hope now for GM, which did not get the benefits of new management, and seems likely to rehash the same mistakes that it has made for decades. It appears that the feds gave up, and just felt thankful that the crash landing will occur at a time when the economy can better handle it.

    I’ve about given up on GM at this point. If it fails down the road, at least the economy should have recovered enough that a company with better management will feel confident enough to pick up the good parts.

  • avatar
    CamaroKid

    The CR survey is quite good because the number of respondents is large (hundreds of thousands).

    Uh no. Has as been basically documented over and over above; the CR survey is better then nothing.. but just barely.

    The problem with the hundreds of thousands of responses is that they are spread over hundreds of sub-models and then the thousands of responses per sub-model are spread over several years… so now your hundreds of thousands of responses is really “hundreds of responses”

    So now you are taking a sample size of a couple hundred at best and making a decision bases on fractions of fractions of issues… Even as you noted a few posts back… we have no idea if these are high mileage cars, driven hard and put away wet.. or if they are cream puffs used only to drive Grand-Kids to Sunday School.

    We see statistical anomalies all over the place… and read the fine print.. CR will “bring forward” scores and ratings if a car has a “long track record” of being really good or really bad… There is a word for that… Oh ya.. Bias.

    You will be hard pressed to find a reputable polling company that will not release its raw data for review and scrutiny… Not CR… Why?

    I don’t think there is a conspiracy… What I do think is that CR likes subscribers and likes to keep subscribers… So when CR sees that a majority of Subscribers drive and like a particular brand of car there is a natural tendency to “play to your base”.

    And worse then that, the CR result seems to be done in “brain dead” mode…. if you know that two cars use the same engines, transmissions, are built on the same line, in the same plant (all information that anyone with access to google can figure out) why not lump their poll results together?

    Why are there SO many obvious… I mean in your face “WTF”… scores every year? I have yet to see a single CR sheet where I couldn’t at a glance point to at least 8 sub-models and say “that can’t be right, that makes no sense”

    Every year when I see this survey all it convinces me is that CR has no business rating cars. Their editors are unable to do even basic homework and their survey is little more then a random number generator.

  • avatar
    ponchoman49

    Quote:skygreenleopard :
    October 27th, 2009 at 5:19 pm

    David Holzman:

    I second this. There are some very mediocre Japanese cars. I don’t hear anyone bragging about their Mitsubishi.

    While that’s a valid point, let’s strike a deal. You can make fun of Mitsubishi and Suzuki all you want (#6 and #13 on the list). They’re admittedly mediocre cars.

    I get to make fun of people who bought Buicks, Lincolns, Pontiacs, Chevies, GMCs, Saturns, Jeeps, Dodges, Cadillacs, and Chryslers (#19-#33).

    They I’ll resell my Integra for several thousand more than a same-year Taurus and laugh all the way to the bank.

    Actually I’ll be laughing at you because you paid thousands more for that Acura to begin with getting you no further ahead whatsoever. The point of purchasing that Chevy was to save thousnds of dollars compared to the over-hyped Asian rivals at purchase time and get a lower car payment, lower insurance costs, have easy access to dealerships if and when repairs are necessary and far easier supplies of normal everyday tires, brakes, oil filters and other parts if the need arose. The old argument of greater resale value at trade is very seldom valid when you add up the greater purchasing costs, greater insurance on hot Japanese fad vehicles like the Camry and Accord, harder to obtain parts like my parents best long time friends with there 2007 camry that has needed no less than 3 replacement transmissions that took months to get in stock making there life difficult to say the least. Sure they are going ot get back more trade in compared to my crappy Chevy. Thats because they have way more invested in that POS than I do.

  • avatar
    Pch101

    so now your hundreds of thousands of responses is really “hundreds of responses”

    That’s a large sample size. When you hear Gallup poll results, those are the sorts of numbers that they use.

    CR will “bring forward” scores and ratings if a car has a “long track record” of being really good or really bad

    There’s absolutely no reason to ignore a track record. Only the domestic companies with terrible track records could believe that ignoring track record is a good idea. (It’s preferable to ignore the past when the past isn’t good.)

    So when CR sees that a majority of Subscribers drive and like a particular brand of car there is a natural tendency to “play to your base”.

    They’re just reporting the results. They aren’t “shaping” anything. They’re just tallying the reports of repairs and producing the result. A count is a count, and the domestic counts are worse.

    Why are there SO many obvious… I mean in your face “WTF”… scores every year?

    They’re aren’t. Name some specific examples, and show how they differ from other surveys such as JDP.

  • avatar
    Bunter1

    A few thoughts on JDP VDS vs CR.

    CRs survey is a reliability survey (BTW average samples per model year approx 600-700).

    JDP VDS is a broad survey that includes reliability data. Reliabiltiy data is covered in only 2 of the 7 or 8 major categories they look at (they happened to publish this info back in 04 or so). Two categories in particular, NVH and Safety, tend IMO to drive the scores of luxury vehicle brands up (lower numerical) and drive down the scores of brands that focus on smaller less expensive vehilces.

    This doesn’t make it a bad survey, it simply is trying to present a lot of data to its customers, the automakers. The VDS composite score is pretty meaningless as a tool for the public, IMO.

    I’m afraid I can’t buy the idea that approx. 600 Caravan owners, 600 Sedona owners and 600 Sienna owners, on the average will interpret the CR survey questions so differently that it will skew the results significantly.

    As to the alledged low participation in the CR survey… they get approx. 1M respondants out of what is it, 5M subscribers? 20% response.
    Accept only those with cars newer than 10 years old can participate.
    Probably looking at between 35-45% participation of those eligible.
    I doubt the “young and angry” are the bulk of this group.

    Geeber and others.

    Suspect you are right, I’ve seen Domestic buyers put up with a lot of crap and Toyonda buyers switch for pretty small beer. Tough to prove either way.

    Bunter

  • avatar
    Bunter1

    BTW-Last year GM had ~50% average or above in CRs survey, this year it’s lower 40s.

    Score
    Consumer perception: 1
    Bob Lutz: 0

    Chuckle.

    Bunter

  • avatar
    wsn

    # Michael Karesh :
    October 27th, 2009 at 10:27 pm

    CTS V6 RWD: 57% much worse than average

    CTS V6 AWD: 41% worse than average

    16% is a large difference in terms of CR ratings–it gets you from a half-black to a full-black dot.

    Looking at the various dots, the big differences between these cars appear to be in the areas of body hardware and brakes–not the drivetrain.

    That’s entirely reasonable. The two models have different weight distribution, different driving dynamics. So even the same mechanical components are subject to different levels of uses. Not to mention the two models may have different quality assurance teams.

    It seems Michael Karesh and many other poster cannot get past the “more component = more problems” mentality.

    Theoretically yes. The RWD CTS may have 5 million potential problems; the AWD CTS may have 6 million potential problems. But after QA and redesign and even some luck, the AWD version isn’t necessarily worse than the RWD version.

  • avatar
    AggieKnight

    @don1967 – “I seriously doubt that adding All-Wheel-Drive or a Hybrid powertrain makes any car more reliable. It likely just makes the owner more enthusiastic when filling out surveys.”

    From a simplistic point of view, you are probably right. The parts themselves do not necessarily make the car more reliable. However, there can be a variety of factors related to the production process that would indeed make them more as a whole. Different shifts, different inspection teams, build quality on the parts that are different, better engineering process for AWD, those are just a few off the top of my head. Given five minutes, I could probably come up with a several dozen factors that could play into the results. It is entirely possible that while mechanically AWD should not be more reliable, that does not mean that it isn’t.

  • avatar
    ronin

    Just want to know when Ford will compensate me for my 1970 Cleveland block Cougar whose side panels rusted to nothing in 3 years, for my brother’s Pinto whose brackets would no longer support the gas tank, for my parents’ new Torino which became the poster child for a new state lemon law.

    This is at the time when they were telling us that their quality approvided.

    Stand by your promise and maybe I’ll believe you again.

    The Hyundai I currently have is in infinitely better shape after 5 years than any of the last Fords we owned during the 70s. So I and my children are happy.

    Come back, Ford, when you make up to me all the money I lost on you.

  • avatar
    CamaroKid

    There’s absolutely no reason to ignore a track record. Only the domestic companies with terrible track records could believe that ignoring track record is a good idea. (It’s preferable to ignore the past when the past isn’t good.)

    Yes there is, when you give a score for a sub-model for a specific year, and that score is based on a “carry forward” then there is a word for that too… its called Bull****.

    I won’t argue that as a “block” most Japanese cars are better built then most American cars, and have been better for years and years now, that is pretty much a given… The issue is that these surveys (both JDP and CR) have so much bias and BS that they are all but worthless.

    You and I both know people that live their lives based on CR surveys and they will pick a 99 4Runner over a 2000 4Runner ’cause the 99 has a full red circle and the 2000 has a half red circle

    A sample of 100 is significant if you are asking do you vote R or D… a sample of 100 is NOT significant if you ask did the engine work perfectly or not? If 99.5% of people don’t have engine issues… there is a very strong likelihood that even a directed survey will miss the 1 or 2 that did have a failure.

    Again this is fractions of fractions… any and all information is lost in statistical static. You are asking a sample size of 100 to tell you the difference between a 99.8% problem free rate and a 99.0% problem free rate. Over 100,000 cars that difference is huge… over 100 it is easily missed.

    This is exactly why JDP and CR contain so many anomalies. When you sample 100 people with the question.. Did you have any engine problems? and get answers of 0, 1 and 2… those answers tell you nothing.

  • avatar
    Steve-O

    ronin: Have You Driven a Ford Lately?? ;)

    But seriously, I can understand why the money lost on a POS in the past will make you unlikely to darken a Ford dealership’s door today. But for the purposes of discussing 21st century car reliability, I think it’s safe to say that any horror stories from 30-40 YEARS back in the day aren’t so relevant.

  • avatar
    Pch101

    when you give a score for a sub-model for a specific year, and that score is based on a “carry forward” then there is a word for that too… its called Bull****.

    That is not what CR does. It seems that you’re not familiar with what they do.

    There are three things at work here: the CR reliability survey, CR’s recommendations and expected reliability.

    The CR reliability survey comes from survey respondents. Respondents are asked if they had problems with their cars in several categories, and they answer in the affirmative or negative. It is not an opinion of whether they like the car, hate America, etc. but just a matter of identifying whether the car had issues in any of the various categories.

    Then there are CR’s recommendations. CR tests the cars for performance, handling, utility, etc. and makes a judgment call of whether their testers like it. If they like the vehicle and the reliability score is average or above, they will recommend it. If CR likes the vehicle but the reliability score is below average or worse, they will comment on the things that they like but will not recommend it.

    Then there is anticipated reliability. For most cars, they will not recommend new models because there is no track record. But until recently, they were willing to recommend Toyotas whose previous models had good track records. They have since dropped Toyota due to the V-6 sludging issue, but have continued to do this with Honda.

    All of that makes perfect sense. There is no “bias” whatsoever evident in using a demonstrable track record to formulate recommendations. It would be foolish for them to do otherwise.

    The domestic fans demand that the rest of us ignore useful data in making decisions. Ignorance is bliss in Detroit, I guess. If you want bias, you’ll find it among those who want us to ignore relevant information because the facts don’t make for a good story.

  • avatar
    CamaroKid

    That is not what CR does. It seems that you’re not familiar with what they do.

    I am very familiar, and have been laughing at CR surveys for years.

    OK maybe “bias” is the wrong word since that implies something “unethical”. “incompetent” might be a better word. “Impossible Task” is probably most fair.

    You continue to ignore the core problem that fundamental with the JDP or the CR surveys… When you are looking for differences in fractions of fractions of differences in percentages you need a sample that is MUCH bigger then a few hundred.

    Can you use the CR (and JDP) surveys to make blanket statements that, as a group, Corollas are better built then Cobalts? For sure… (well duh I would say, who doesn’t know that) But can you say that the reliability of a V6 99 4Runner limited is better then a V6 2000 4Runner Limted? No way.

    And yet, that is exactly what the report attempts to do. And that is why it is useless.. You can’t slice the data into such small groups and claim any degree of reliability.

  • avatar
    Pch101

    When you are looking for differences in fractions of fractions of differences in percentages you need a sample that is MUCH bigger then a few hundred.

    Sorry, but you’re just wrong. The sample sizes are large enough to be statistically significant with a reasonable confidence level. Good luck finding a reliable source grounded in statistical analysis to support your position.

    But can you say that the reliability of a V6 99 4Runner limited is better then a V6 2000 4Runner Limted? No way. And that is why it is useless.

    Their surveys are quite useful, as they mirror real world outcomes quite accurately. Trying to use exceptions (real or imagined) as rules actually proves the point — the exceptions are just that.

    Fans of the domestics don’t care to admit it, but the fans are just wrong on the facts. Unfortunately, all the domestic fans to support themselves are anecdotes that are offered to support their own, er, biases.

    The fact is that there isn’t a single useful source that shows that GM and Chrysler are on par with the best of the transplants. Not one. The pro-domestic camp has no real arguments, so they lob around a mix of accusations and hyperbole in the hope that something sticks.

  • avatar
    Steven02

    @Michael Karesh

    For the Lambdas in particular, do you know what average is?

  • avatar
    Bunter1

    1.4 million surveyed.
    Approx. 250 models.
    10 year spread.

    1.4M/(250*10)=560 data points average per vehicle.

    Actually a bit higher as many vehicles have only a few years in their run.

    Far more data points than JDP or TD per vehicle.

    Problems under a few percentage points could easily be missed with only 25-50 minimum samples.

    With 5-600 on the average not much will slip through.

    Camarokid-It is actually pretty easy to see when some problems, say the transmission problems on some V6 Hondas, show up and when they were fixed in CRs data.
    You can also see that GM never addressed the front drivetrain issues in ~10years of the previous Silverado.

    This is very usefull information, year-by-year, for anyone willing to look at it.

    Cheerio,

    Bunter

  • avatar
    CamaroKid

    Sorry, but you’re just wrong. The sample sizes are large enough to be statistically significant with a reasonable confidence level. Good luck finding a reliable source grounded in statistical analysis to support your position.

    You will have just as much luck finding someone to tell you that the sample sizes are large enough… As I posted (and was confirmed by others) A sample of 100 is plenty big to compare do you prefer brand X of gas vs brand Y.

    We are not comparing that… You are comparing failure rates of 0.01% vs 0.05% out of a population of 10’s or even hundreds of thousands and sorry, you are not going to find a reliable source grounded in statistical analysis to support your position either.

    I am not a domestic fan, as you keep posting… I have said over and over that Japanese cars as a group, are better then American cars (as a group) if CR stuck to that, and reported that, they would be fine. I’m the guy who pointed out that NorthStar engines are crap… and have been for over 10 years now… and CR reports that they have better then average reliability… Ya great confidence level in the reporting there.

    But you fail to recognized what every statistician in the world knows, every time you slice a survey you pickup statistical errors.. and the CR Survey has been sliced so many times that when you get down to it.. it is one big error.

  • avatar
    Steven02

    @Pch101
    Exceptions are a real big part of statistics that need to be explained. But, it seems that people are able to point out several exceptions for every year on the survey. At that point, you can’t classify them as exceptions.

    @Bunter1
    Your division assumes that all models are equally sold. This is simply not the case. For an example, there are far more Civics on the road than BMW 3 series. For 1200 respondents for the 2 cars, you may get 1100 Civics, and 100 3 series. Not exactly optimal.

    I think what most people are missing here is the statical analysis has been changed a few time at attempts on segregating the competition, red dots to black dots. Say you have a car that has 1% failures. And another with 3% failures. One is above average, one is worse than average, that is the rating from CR. Does it mean that you shouldn’t look at the 3% car? Does it not mean that both of them are actually quite good cars?

    The problem with consumer reports ratings is that you can have a very good reliable car, but still be worse than average. Say the rate of failures is 1% as an average. A car you are looking at is 1.21% failures. Does that mean that this car is worse than average? In consumer reports, it does. Is this what is happening, no one really knows since they don’t report the actual data. But .21% or less can me a lot in these ratings, but not in the real world.

    Also, the definition of failures is not very well defined, which is another problem with CR’s method.

    You should read Michael’s document about this. It is pretty interesting.

    http://www.truedelta.com/pieces/newdots.php
    http://www.truedelta.com/pieces/shortcomings.php

  • avatar
    geeber

    CamaroKid: I’m the guy who pointed out that NorthStar engines are crap… and have been for over 10 years now… and CR reports that they have better then average reliability… Ya great confidence level in the reporting there.

    Pch101 gave a thorough and very good explanation of the problem with the survey results for cars equipped with Northstars.

    Most of these cars are purchased brand-new by older people who, on average, don’t rack up all that many miles in one year. They certainly drive far fewer miles than the typical American.

    I don’t see many people under the age of 50 flocking to buy Buick Lucerne Supers or virtually any Cadillac equipped with the Northstar V-8.

    As a result, they don’t experience the engine failure within the survey’s timeframes. Plus, being older, they are more likely to have the mindset that, as the odometer nears the 100,000-mile mark, it’s time for a new car. So they trade before the engine failure occurs. (I see this with my parents, who are loyal GM owners.)

    Given the low resale value of these cars, they end up being purchased by people who are less likely to subscribe to Consumer Reports. At least, that is based on the type of people that I see driving old Northstar-equipped Cadillacs around here.

  • avatar
    VanillaDude

    There has become a blatant bigotry against US made cars. Just how many Japanese auto drivers would switch to Ford or any other American made car based on the evidence we’ve reported?

    Few. That’s a fact. What we have are drivers who favor Japanese cars partly because they are not American. They have an anti-American bias, and would not give US makes a try even if US Auto makers would continue to demonstrate their ability to meet world-class quality.

    We have a generation of people who are not open minded about American cars. I know dozens. They wouldn’t drive an American car even if they were as good as their Honda or Toyota. They feel that their foreign cars enhances their status.

    Just as it was tough to get drivers to buy Japanese cars originally, it will be just as difficult to get those same people into American cars.

    Why does this matter? Because assembling cars in the US is not the same. Mexican workers don’t consider Ford a Mexican company because Ford is paying them to assemble cars in Mexico. So just how willfully blind are Americans to call their Japanese cars, “American”? With the logic these people use, plantations were African American operations because blacks picked the cotton, and that is utter nonsense.

    We have closed minded Japanese auto owners who look down on American cars. We have more than one generation of them. So, regardless of how excellent Ford makes cars, these people will not buy one, regardless of if their US wages end up in Tokyo. Instead they will justify their auto purchases with the fraction of the Japanese car profits, returning to their neighbors for helping assemble and wash their cars.

  • avatar
    Pch101

    You are comparing failure rates of 0.01% vs 0.05% out of a population of 10’s or even hundreds of thousands and sorry, you are not going to find a reliable source grounded in statistical analysis to support your position either.

    Again, this is wrong, on a couple of counts. For one, the failure rates are higher than that. For another, a sample size of 100 is going to be statistically significant, regardless of population size. Any statistical text will tell you this.

    But, it seems that people are able to point out several exceptions for every year on the survey.

    I keep hearing allusions to unproven anecdotes, but not much more than that. I’ve looked at enough of the data to see that it is generally consistent and comparable to those of other surveys.

    Once again — there is not a single credible survey that shows GM and Chrysler to be on par with Honda and Toyota. Not one. There is no avoiding this fact.

    Most of these cars are purchased brand-new by older people who, on average, don’t rack up all that many miles in one year. They certainly drive far fewer miles than the typical American.

    Here’s the funny part — the one obvious flaw of the Consumer Reports and JD Power surveys is that the results are not mileage-adjusted, and yet nobody complains about this.

    Since we all know that usage reduces the longevity of the product and we also know that mileage does not impact all vehicle nameplates equally, you would think that this disparity would be accounted for in the reports, yet it isn’t.

  • avatar
    ChevyIIfan

    +1 to VanillaDude

  • avatar
    PeteMoran

    @ VanillaDude

    Just as it was tough to get drivers to buy Japanese cars originally, it will be just as difficult to get those same people into American cars.

    The Japanese knew what they had to do and the executed on it. GM/Ford/Chrysler knew what they had to do and wiped themselves out. The CR/JDP/TrueDelta data points the way.

    We have closed minded Japanese auto owners who look down on American cars.

    Which ones are you talking about? The ones that buy US made but Japanese parented cars like the Camry, Corolla, Accord or Civic (and others)??????

    Your post is a call to some form of patriotism that you feel isn’t served already by people buying US made Toyotas or Hondas. You insult people consciously buying US made already. US made that is good value and high quality.

    Damn the “perception gap” because the customers are stupid bigots.

  • avatar
    Maxb49

    Again, this is wrong, on a couple of counts. For one, the failure rates are higher than that. For another, a sample size of 100 is going to be statistically significant, regardless of population size. Any statistical text will tell you this.

    The Consumer Reports defense squad is trying to talk about statistical siginicance? You can’t talk about statistical significance because you haven’t even gotten past the PROBLEM of Consumer Reports external invalidity. The “reports” are plagued by a self selection bias and thus not a random sample!

  • avatar
    jmo

    US Auto makers would continue to demonstrate their ability to meet world-class quality.

    No! They don’t need to meet, they need to exceed. Being just as good isn’t going to get them anywhere.

  • avatar
    Happy_Endings

    They wouldn’t drive an American car even if they were as good as their Honda or Toyota.

    Ah, the old “it’s the consumers fault” argument. The reason these people stopped buying domestics can be completely ignored because these people have a “bias” against American cars. Actually making a better car than the transplants is way too hard to do. So you complain that the customers are “anti-American” or whatever other argument you make.

    Just as it was tough to get drivers to buy Japanese cars originally, it will be just as difficult to get those same people into American cars.

    There is a way to get these customers back. It’s called “making a better car than the competition”. I know, it’s a difficult concept. Making a competitive car won’t cut it. If you are a Camry owner and have never had a problem with it, why would you buy a Malibu when it is only just as good? Why would go back to a domestic when they may have made you a crappy car in the past?

    So, regardless of how excellent Ford makes cars, these people will not buy one, regardless of if their US wages end up in Tokyo.

    Every single one of these companies are global corporations. They aren’t loyal to any single country or the country they happen to be headquartered in, just whatever country gives them the most profit at that time. In what countries are pretty much all car companies investing heavily in right now? CIBR, China, India, Brazil, and Russia.

  • avatar
    Pch101

    You can’t talk about statistical significance because you haven’t even gotten past the PROBLEM of Consumer Reports external invalidity. The “reports” are plagued by a self selection bias and thus not a random sample!

    I previously addressed this, and again, you’re wrong.

    The reliability survey is a survey about just that — the reliability of vehicles. Not the opinions of the vehicle owners, but operating data about the vehicles themselves.

    For someone to believe your thesis, one would have to think that Consumer Reports readers either (a) lie more often than other people with respect to their cars or (b) buy more domestic lemons than does the general population.

    The domestic fans have no ground to stand on, as everything they believe comes down to the need for some grand conspiracy for their points to be accurate. The fact that the CR survey correlates pretty well with other such surveys is a strong hint that the CR survey is generally accurate, and that most of the domestics don’t cut it.

    It’s no surprise to me — all of that time spent whining distracts the Detroit cheerleaders from making improvements. It’s not shocking that they wouldn’t bother fixing anything when they don’t think that anything needs fixing. Just remember — every time you argue that they’re equal, you just alienate the consumer who knows that they aren’t, and reduce your credibility at the same time.

  • avatar
    Maxb49

    For someone to believe your thesis, one would have to think that Consumer Reports readers either (a) lie more often than other people with respect to their cars or (b) buy more domestic lemons than does the general population.

    One of my degrees is in statistics. Clearly, yours is not, evidenced by your failure to understand why a selection bias is a fatal error to any statistical study. The number of reports sent in to the Consumers Union on a given model is far less than the actual numbers of models produced. The reports mailed in to CR are called a “sample”, and the total number of cars of a given model manufactured is the “population”. The sample size is a minor fraction of the sample size. Consumer Reports holds that its reliability data generalizes to the population of a each and every model on which they report.

    Here’s the flaw in your reasoning: To make accurate generalizations from a sample to a population, you have to assume that the reports are normally distributed. The way to assume a normal distribution is with a random sample. Consumer Reports does not have a random sample because

    1.) The only people included in their study are those individuals who subscribe to Consumer Reports.

    2.) From this, the sample is not drawn randomly. A questionnaire is sent out to subscribers, who choose whether or not to participate in the survey. This is called a self selection bias. It distorts the statistical data by allowing the sample to choose itself.

    3.) Consumer Reports then draws statistical inferences from this nonrepresentative, non random sample.

    You make assumptions that everyone is telling the truth about their car at the same rate as the overall population. This is an irrelevant assumption. The lack of a random sample leads to the problem that the population of cars themselves are not faithfully and accurately represented in the population.

    You cannot overcome a selection bias. When the study is not a sampled randomly, the study is a no go.

    Here are some links on selection bias, and more problems associated with the issue than dealt with in this post:

    http://en.wikipedia.org/wiki/Self-selection

    http://www.skepdic.com/selectionbias.html

  • avatar
    Pch101

    The number of reports sent in to the Consumers Union on a given model is far less than the actual numbers of models produced.

    Right. And when Gallup does a poll of voters, you must believe that they contact all 100+ million of them.

    It’s called a sample for a reason — because samples can be used to estimate the results of the population. It isn’t necessary to ask every single vehicle owner about his car to get a useful result.

    The only people included in their study are those individuals who subscribe to Consumer Reports.

    And I have noted ad nauseum, unless you believe that these owners (a) are uniquely dishonest about the craptasticness of their GM and Chrysler vehicles and uniquely dishonest about their Toyotas and Hondas, and/or you think that CR buyers are blessed with uniquely good Toyotas and all of the lemons that GM produces, then this doesn’t matter very much.

    It’s ridiculous to think that CR buyers of domestics are the only buyers who have problems with them. Again, it’s a survey about the reliability of the **vehicles**, not the opinions of the humans who own them. It’s a quantitative survey, not an opinion poll.

  • avatar
    Maxb49

    Right. And when Gallup does a poll of voters, you must believe that they contact all 100+ million of them.

    It’s called a sample for a reason — because samples can be used to estimate the results of the population. It isn’t necessary to ask every single vehicle owner about his car to get a useful result.

    Hold it right there. It’s time to brush up on your basic statistics before you continue to embarrass yourself any further.

    The difference between Gallup/Zogby/Rassmussen/etc – credible polling agencies – and
    Consumer Reports has to do with their sampling strategy. Those polling firms take a RANDOM SAMPLE. Consumer Reports does NOT. End of story.

    You really don’t know what the hell you’re talking about. Look up: random sample, standard error, selection bias, statistical inferencing. I’m not going to wipe your a** for you.

    And I have noted ad nauseum, unless you believe that these owners…

    Reiterating the same ignorant bull*** doesn’t make it true. You incorrectly assume a normal distribution of the sample size. That’s absolutely frickin impossible given that the sample size is biased.

    …and they wonder why America has problems with science and math?

  • avatar
    Pch101

    Those polling firms take a RANDOM SAMPLE.

    Actually, they don’t. They only survey people with listed telephone numbers. That is obviously going to exclude the sorts of people who choose to have unlisted numbers or only have cell phones. So no, you are incorrect.

    In any case, you again miss the point, so I will provide an analogy. Let’s suppose that we did a reliability survey of vehicles, but we limited the respondents to those who like to watch the television show “Friends.” That would obviously exclude a lot of people (myself included) but it is highly unlikely that the result of the survey would be wrong, because there is no correlation between one’s propensity to watch that program with the durability of the fuel injectors in their cars.

    CR is a mainstream magazine with several million subscribers. I know that you wish to believe that it is some sort of anti-American cult, but you are wrong. It is clear from the survey that the readership buys all sorts of cars. There is no reason to believe that they would obtain a significantly different result than a completely random sample.

    And sure enough, JD Power’s VDS which is more random produces very similar outcomes. In this case, randomness doesn’t mean much, because the variable in question is not meaningful.

    I’ve challenged you to present a legitimate source that shows domestics to be on par with the best transplants. You obviously can’t.

    And I have taken graduate-level stats, so I am familiar with the topic. But thanks for asking.

    That’s absolutely frickin impossible given that the sample size is biased.

    You don’t know that, you’re just assuming it because of your wishful thinking.

    CR excludes vehicles for which the sample sizes are too small. They get over a million responses, which allows for pretty good representation of a lot of vehicles, obviously.

  • avatar
    Maxb49

    Actually, they don’t. They only survey people with listed telephone numbers. That is obviously going to exclude the sorts of people who choose to have unlisted numbers or only have cell phones. So no, you are incorrect.

    Okay, actually they do. The numbers are generated by random number generators. Keep embarrassing yourself!

    In any case, you again miss the point, so I will provide an analogy.

    Your point is that statistical studies based on biased samples are valid. You’re absolutely incorrect.

    CR is a mainstream magazine with several million subscribers. I know that you wish to believe that it is some sort of anti-American cult, but you are wrong.

    I didn’t assert that they were an anti-American cult – stop trying to shift the debate away from your failure to understand basic statistics by shoving words in my mouth.

    And I have taken graduate-level stats, so I am familiar with the topic. But thanks for asking.

    I’m sure you’re proud of that D-. That is, if you passed.

    You don’t know that, you’re just assuming it because of your wishful thinking.

    Everyone with a brain in their head knows that it’s biased. It’s called self-selection bias. Do you know what constitutes a self selection bias? Explain a self-selection bias to the readers of this blog if you hope to earn back a shred of credibility. Then we’ll talk.

  • avatar
    Pch101

    It’s called self-selection bias.

    All surveys have self-selection bias.

    If a pollster calls, I can hang up and not participate.

    If it arrives in the mail, I can ignore it.

    If an email shows up, I can ignore that, too.

    The respondents ultimately have to decide to respond. For your position to be correct, you would have to presume some vast conspiracy in which many of the domestic owners behave in one fashion and the transplant owners behave in another. In other words, some fantasy world that doesn’t exist.

    Since you keep ignoring this, I am going to challenge you again — provide a credible survey that shows that domestics are on par with the best transplants.

    I know that you can’t do it, because such a survey doesn’t exist. There isn’t a single valid quantitative source that you can point to that would support your position. You just like the domestics and keep hoping that someone will believe you, even though your positions are based upon hope and thin air.

  • avatar
    CamaroKid

    Pch101 gave a thorough and very good explanation of the problem with the survey results for cars equipped with Northstars

    Which was EXACTLY my point. You need to know this, and consider that, before you read the survey, because it tells you nothing.

    The reason a known unreliable engine gets a good score is because owners of these cars never drive them… WTF How does the survey help me if I plan to, you know, actually drive the car?

    Lets say the survey says that Escalades have a really bad version of the LS motor… is that because the motor is a pile of crap or be cause Escalades are driven harder and farther then a Tahoe?

    Here is an analogy of what Consumer Reports does…

    Polling is usually like this…. Gallup wants to know who is going to win the next election… They phone up 1500 (or so) people and ask, who are you voting for and even with that small of a sample they can predict within a small percentage who would have won the national election if the vote was taken that day…

    Consumer Reports doesn’t stop there though… they continue to slice and dice the numbers into meaningless minutia… Its like taking the above poll and trying to predict how many people on your block were going to vote for McCain… You know on an average block that about 46% did… but in reality very few blocks are average… most blocks vote 80 or even 90% one way or another… When CR slices the 100,000 responses down to 100’s and takes those and spreads them over 9 or 10 “issue” points… you are left with nothing.

    And then they don’t stop there… In an attempt to make the chart easy to read they roll up the scores into a rating system that is meaningless… really good, good, ok, bad, really bad… What is a “really good” score? One expected failure out of 10,000? Why not just post the failure percentages and let us see just how really good the engine on a Lexus is or how really bad the transmission is on a Chysler minivan…

    Since as Pch already thoroughly explanated that this report is useless unless you know that a car is only driven a few miles. I can add we also don’t know what a really bad failure is… Is an engine failure a $5 O-ring seal leak or a $5000 head gasket failure. Does CR count them the same? We don’t know.

    Imagine that I buy an extended waranty based on this survey only to discover that the “black circle” score was do to a $50 part that the manufacture has resolved.

  • avatar
    Pch101

    How does the survey help me if I plan to, you know, actually drive the car?

    It’s a survey of reliability of those who own the cars. If you are a typical owner, then your experience could be expected to be typical. If you aren’t typical, then you may have to expect an atypical result. That’s true of anything, isn’t it?

    When CR slices the 100,000 responses down to 100’s and takes those and spreads them over 9 or 10 “issue” points… you are left with nothing.

    False. The hundreds of responses aren’t “spread” over anything — those respondents answer **all** of the reliability questions for a given vehicle, not just one of them. You really don’t seem to understand how the survey works.

    In an attempt to make the chart easy to read they roll up the scores into a rating system that is meaningless

    The chart mirrors the questions that are answered by the respondents. That list of areas that you see in CR is the same list that respondents address in their replies. It’s fairly clear that you’ve never actually seen the survey, so I would suggest that you not attempt to describe something of which you lack familiarity.

    What is a “really good” score? One expected failure out of 10,000?

    The percentage equivalents are provided. I reiterate my comments about your lack of knowledge in this area.

    Since as Pch already thoroughly explanated that this report is useless unless you know that a car is only driven a few miles.

    Don’t put words in my mouth. I made the comment that the results are not mileage adjusted and that demographics influences vehicle mileage. That does not make the survey “useless” by any means, but illustrates the potential limitations.

    Imagine that I buy an extended waranty based on this survey only to discover that the “black circle” score was do to a $50 part that the manufacture has resolved.

    Since when does using Consumer Reports relieve you of performing other research? As a buyer, you should expect to check more than one source.

    It sounds as if you wish to critique these surveys because none of them relieves you of the responsibility of looking at anything else. In this case, the problem is with unreasonable expectations, not with the survey, which will have both its inherent advantages and disadvantages.

  • avatar
    CamaroKid

    If you are a typical owner, then your experience could be expected to be typical. If you aren’t typical, then you may have to expect an atypical result. That’s true of anything, isn’t it?

    So we can agree that it isn’t scientific at all then… How do you compare a car that is “typically” drive 30,000 miles a year with one that is driven 3,000 miles per year.

    How does a reader know if they are “typical”?

    Again I understand exactly how this works… and we have had two people with post graduate statistics under their belts agree that the sampling is bunk.

    Notice how like CR, you just dodge all of the questions I pose…

    Why not post the raw data? Why not post the numbers in terms of failures per miles driven? Why not post the numbers in terms of $ per miles driven… This sort of data would be very useful… Red, Black and White circles based on Statistically irrelevant sampling is not.

    That does not make the survey “useless” by any means, but illustrates the potential limitations.

    LOL, what the heck do you think I have been saying for the last umpteen posts. Ask a statistician about “Potential limitations” and they will give a percentage confidence factor 9 time out of 10. CR surveys have no confidence factor and have “obvious” problems each and every year. If Gallup had “limitations” and obvious errors on each and every presidential poll for the last 30 odd years do you think they would still be in business… and yet CR does exactly this.

    Since when does using Consumer Reports relieve you of performing other research? As a buyer, you should expect to check more than one source.

    Ever talk to a CR reader? They never check more then one source. They live by this stuff. Look at this thread, we have people who would rather sell their kids then admit that the CR Survey has some serious issues and errors.

  • avatar
    PeteMoran

    @ CamaroKid

    Why not post the numbers in terms of failures per miles driven?

    Should they be “city” or “highway” miles? In higher or lower temperatures?

    Since there’s nearly no way to correct for that, the only useful measurement is the age of the vehicle.

    Either way, in market research, the only “test” of one methodology is another methodology of a slightly different design.

    As Pch101 points out, that is the JDP alternative, which fairly well parallels CR which tends to strengthen the claim of (statistical) significance for both sets of results.

  • avatar
    CamaroKid

    Should they be “city” or “highway” miles? In higher or lower temperatures?

    Since there’s nearly no way to correct for that

    Thank you, this is EXACTLY my point. Since we have no idea of what life the car has lived polling their owners tells us nothing. Since there is no way to correct for that, we need understand that this poll is less then scientific… at best it is a “curiosity”… That was my point.

    As Pch101 points out, that is the JDP alternative, which fairly well parallels CR which tends to strengthen the claim of (statistical) significance for both sets of results.

    Again this was my point…generally… fairly well… parallels.. what ever word you want to use all of these polls generally say that Japanese cars tend to be better built then American cars… (and on that I agree 100%) The problem is where you zoom in so close that you have no idea what is going on… and look what happens when you “zoom in” on JDP and CR.. you find cars all over the place that one poll says is rock solid and the other says is crap. As we have seen over and over this even occures within the same survey. We can seen two ‘clone’ cars made on the same line, same worker, using the same parts and one gets a full red circle and the clone gets a half red (or worse)

    This can ONLY occur if one or both poll aren’t statistically relevant.

    Which was my point and has been supported now by several bloggers…

    This is kinda funny really.. we have people who want to have it both ways… they want to claim that the survey is scientific and statistically relevant while they point out one example after another that requires a non-scientific and statistically anomalous explanation.

    Its scientific as long a you accept that it isn’t an apples to apples comparison in terms of miles, condition, units sold, demographics, geographies, cost to fix, dealer experience, etc etc etc… or in other words it isn’t at all scientific.

    You can’t have it both ways.

  • avatar
    PeteMoran

    Since we have no idea of what life the car has lived polling their owners tells us nothing.

    A large enough sample size will cure that. You’ll get some from the north-east, some from Florida, some from LA, some from fly-over. It won’t be something you have to control for.

    CR publish their error and claim of significance no?

    we need understand that this poll is less then scientific …… or in other words it isn’t at all scientific

    Scientific research and market research/polling should never be compared. The scientific method requires the concept of a control.

    These CR/JDP market research efforts are far from a simple “curiosity” however.

  • avatar
    CamaroKid

    A large enough sample size will cure that. You’ll get some from the north-east, some from Florida, some from LA, some from fly-over. It won’t be something you have to control for.

    We have already established that the CR survey lacks the needed sample size at the year/sub-model/problem area level… even CR admits that for some cars the sample is as small as 100 cars. By your own and others admissions, the sample size is small enough that problem engines that are driven by “older” people who never drive their cars very much get a better score then they would if a proper and statistically relevant sample was taken. My point is if you take a PROPER sample you aren’t constantly having to go to say “ya that does look odd, I guess those drivers aren’t typical”

    CR publish their error and claim of significance no?

    Yes you are right, they don’t. And unlike every other polling company on the planet they refuse to release their raw data. If it so clean and so scientific, why hide?

    These CR/JDP market research efforts are far from a simple “curiosity” however

    One more time with feeling… Yes, when you compare everything that GM makes vs everything that Toyota makes they tell you TONS… At that level there are enough responses that the errors start to get lost in the data. Even if you compare at the model level, they are still significant (all Corollas vs all Cobalts for example)

    But if you think that the score of a 02 Corolla VE’s transmission vs a 2005 Cobalt LT’s transmission is telling you anything, you are in for a disappointment. At that level, it is statistical noise.

  • avatar
    Pch101

    We have already established that the CR survey lacks the needed sample size at the year/sub-model/problem area level

    No, that has not been established at all. You have claimed it, but your comments accompanying those claims make it clear that a lot of your impressions about the survey are wrong, and that your assertions are not based upon facts.

    And unlike every other polling company on the planet they refuse to release their raw data.

    No, most polling companies don’t release their data to those who don’t want to pay for it.

    And sorry if I’ve misread this, but I seriously doubt that you’d be willing to compensate them, or even read a free copy, of a compilation of 1 million + surveys, line by line. Give me a break — you’re just looking for excuses to ignore it.

    The fact is that we use polls with a few hundred people surveyed to gauge the sentiment of tens of millions of people, and we do this routinely without much howling from domestic car fans. But when CR catches a much larger proportion of the vehicles sold in this country, the fans have a hissy fit about it despite the fact that their samples are much larger than the norm. (They are certainly larger than JD Power’s pool.) That doesn’t make any sense.

    you find cars all over the place that one poll says is rock solid and the other says is crap.

    If you are going to assert that, then you should compile your data points to prove it.

    I have looked at both, and from what I’ve seen, the JD Power VDS and CR surveys generally correlate with each. It is not at all common for them to produce vastly different results. They are both telling the same basic story: (a) Toyota and Honda lead the pack, (b) GM and Chrysler are generally not very good, (c) “European” and “reliability” are definitely not synonymous and (d) Ford is catching up and may now be equal to Toyota and Honda. Definitely useful, far from useless, and as far as I can tell, pretty damned accurate.

  • avatar
    CamaroKid

    No, that has not been established at all. You have claimed it, but your comments accompanying those claims make it clear that a lot of your impressions about the survey are wrong, and that your assertions are not based upon facts.

    Your right we have just had 4 or 5 blogger, who claim to have studied statistics at a masters level agree that 100 sample points to determine 5 degrees of a fractional percentage difference over a population of 30,000-100,000 is mostly random error.

    No, most polling companies don’t release their data to those who don’t want to pay for it.

    And CR data is NOT available at ANY PRICE. Don’t believe me, ask for the price, I’ll wait.

    They are both telling the same basic story: (a) Toyota and Honda lead the pack, (b) GM and Chrysler are generally not very good, (c) “European” and “reliability” are definitely not synonymous and (d) Ford is catching up and may now be equal to Toyota and Honda. Definitely useful, far from useless, and as far as I can tell, pretty damned accurate.

    This is almost getting comical now… I post that at the “general”, “basic”, “common” level they are accurate, they do say that Ford is getting better and that the Japanese still make very good cars and truck. GM is mid pack at best and Chrysler still sucks… While I post that you continue to ignore the problem..

    THE problem is that they DON’T only publish the above.

    They publish a minutia ratings of a 5 point system of transmission (and other) problems at a smaller then sub-model slice on a year by year basis on next to no data

  • avatar
    Pch101

    They publish a minutia ratings of a 5 point system of transmission (and other) problems at a smaller then sub-model slice on a year by year basis.

    Have you actually read the survey? I honestly can’t believe that you have, as much of what you say is just factually wrong.

    The CR survey has no minutia at all. It uses perhaps a dozen broad categories in total to cover the entire vehicle. That’s all they do.

    The JD Power VDS and IQS do have minutia, but that is for paying clients, not the general public, and that pool is substantially smaller.

    It’s fairly clear that you don’t know anything about the CR survey, and much of what you believe is just wrong. You accuse CR readers of belonging to some bizarro cult, yet it is you who seems fixated on making claims about the survey that just aren’t accurate. Just because it feels good to say stuff doesn’t make it true.

  • avatar
    CamaroKid

    Have you actually read the survey? I honestly can’t believe that you have, as much of what you say is just factually wrong.

    The CR survey has no minutia at all. It uses perhaps a dozen broad categories in total to cover the entire vehicle. That’s all they do.

    I read the survey every year, I’m beginning to wonder if you do though…

    Again, the problem with the survey is that it doesn’t just say Toyota is better then GM… it takes the data and produces a HUGE multi-page chart where it breaks it down by make, model, sub-model, in some cases engine or drive-line specific sub-sub models, breaks that down into about 8 or 9 sub categories and then spreads all that data over 10 years.

    Time to back up your POV, you have yet to do that, where are all of the PhD Statisticians who agree that you can spread data that thin and make prediction about populations in the 30-100,000 range, with 5 different classification schemes, measuring fractions of a percentage differences, based on a sample of 100-500… Sorry, again, at that level, you can’t. My point, CR knows this too, why do they publish this stuff that they know is wrong. Publish numbers that are statistically relevant, it is easy to poll failure rates at the model level that would be accurate, but they don’t do this… odd.

    It’s fairly clear that you don’t know anything about the CR survey, and much of what you believe is just wrong. You accuse CR readers of belonging to some bizarro cult, yet it is you who seems fixated on making claims about the survey that just aren’t accurate. Just because it feels good to say stuff doesn’t make it true.

    You keep posting the same errors and mistakes over and over with no support and no data… I’ll wait.

  • avatar
    geeber

    Pch101: Which was EXACTLY my point. You need to know this, and consider that, before you read the survey, because it tells you nothing.

    The reason a known unreliable engine gets a good score is because owners of these cars never drive them… WTF How does the survey help me if I plan to, you know, actually drive the car?

    What you are basically saying is that Consumer Reports needs to somehow compensate for mileage driven, or say WHEN the expected failure is to occur, or extend the survey results for a greater number of years.

    That is certainly a valid concern.

    But this doesn’t prove that the surveys don’t spot the failure when it does occur, which would happen if survey participants are either covering for their favored marque, or forgetting to list mechanical problems when completing the survey. This is what concerns most people.

    CamaroKid: Thank you, this is EXACTLY my point. Since we have no idea of what life the car has lived polling their owners tells us nothing. Since there is no way to correct for that, we need understand that this poll is less then scientific… at best it is a “curiosity”… That was my point.

    I have yet to see any reliabilty survey that gives that breakdown of information. Basically you are saying that all of these surveys are therefore useless, since they don’t provide this information, so we are back to asking Uncle Joe about his car, or the gang over at GMInsideNews.com about the reliability of various vehicles.

    Somehow, I don’t trust them more than Consumer Reports.

    CamaroKid: This is almost getting comical now… I post that at the “general”, “basic”, “common” level they are accurate, they do say that Ford is getting better and that the Japanese still make very good cars and truck. GM is mid pack at best and Chrysler still sucks… While I post that you continue to ignore the problem..

    THE problem is that they DON’T only publish the above.

    They publish a minutia ratings of a 5 point system of transmission (and other) problems at a smaller then sub-model slice on a year by year basis on next to no data

    I’ve talked to many mechanics, and they say that the magazine’s results are reasonably accurate (not perfect, but at least reasonably accurate), even for the “minutia” ratings.

    For that matter, the magazine has been accurate so far for our two vehicles – a 2003 Honda Accord EX and a Ford Focus SE sedan.

    We buy the Consumer Reports annual auto issue, participate in Mr. Karesh’s TrueDelta program, and I make it a point to ask mechanics which company’s vehicles are the most reliable and best-engineered.

    When the same basic answers from three different sources all line up, you begin to think that they are all on to something.

  • avatar
    Pch101

    When the same basic answers from three different sources all line up, you begin to think that they are all on to something.

    Absolutely. (And you may as well add JD Power to that list.) The domestic drum beaters have one hell of a time explaining that degree of consistency.

    Based upon the current numbers, CR is surveying about 9/10th’s of one percent of all of the passenger cars in the US today. That’s a vastly greater percentage than the proportion of respondents of any other survey that you’ll find, of any kind.

    Nielsen measures the television viewing habits of Americans with about 1250 households. A typical political poll for national US issues will be based upon perhaps 1,000-1,500 people, to cover a base of about 120 million voters. Nobody makes the outlandish claims of “uselessness” about these that are being made here about CR.

    Nobody does a larger survey than Consumer Reports. It covers more respondents, by far, than any other survey in the business.

    If GM and Chrysler want to disprove it, then let them step up to the plate by having them disclose their records of reported repairs, and repairs requested by consumers that they have denied. Since they wish to argue equality, let them prove it. Of course, they won’t, because the truth would just make things look worse for them.

  • avatar
    geeber

    Pch101,

    If GM and Chrysler had put as much energy into improving their vehicle development and quality processes as they do complaining, they would be ranked first in all of the surveys. Same for VW.


Back to TopLeave a Reply

You must be logged in to post a comment.

Subscribe without commenting

Recent Comments

New Car Research

Get a Free Dealer Quote

Staff

  • Contributing Writers

  • Jack Baruth, United States
  • Brendan McAleer, Canada
  • Marcelo De Vasconcellos, Brazil
  • Vojta Dobes, Czech Republic
  • Matthias Gasnier, Australia
  • W. Christian 'Mental' Ward, Abu Dhabi
  • Mark Stevenson, Canada
  • Cameron Aubernon, United States
  • J Emerson, United States