Also, for a large part of the new car buying US population, CR’s reports are highly influential. They certainly have more influence than the enthusiast magazines. Speaking of which, if you correlate the actual reports from the traditional magazines with CR’s reports you will find that they are in agreement more often than not. The enthusiast magazines put ZERO weighting on crash test results or reliability history when they do their Comparison Tests, other than that, the data and write-ups generally have more in common than not. Both CR and Car and Driver found the Saturn Ion wanting. Bias? Not.

Quote:
In all, the survey was sent to almost 6.7 million subscribers in 2006, and we received responses on about 1.3 million vehicles.
http://www.consumerreports.org/cro/cars/maintenance-accessories/consumer-reports-car-reliability-faq-8-06/overview/0608_consumer-reports-carreliability-faq_ov.htm

Just because they refuse to release their method, CR’s credibility is not automatically zero. The Federal Reserve no more releases monetary supply data, does that imply the entire American economy has zero credibility?

http://www.allpar.com/cr.html
“the CR survey may over/understate the reliability of certain cars because the people that own them are not homogeneous. … many people will have a subconscious need to justify their purchase of a Japanese auto over of a domestic one, and they could do this by believing superior reliability is the reason they bought it. Because of cognitive dissonance, they would tend to overlook or downplay anything that would attack this mind-set. We do see many people who vehemently defend Japan’s cars’ reliability and smear that of others.”

Mike P:
March 7th, 2007 at 11:21 am
The scale they have must limit car-owner bias.

From the same article:
A high response rate is the key to validity. Employee survey findings can be questioned when fewer than half of the employees respond. So how many people respond to a Consumer Reports survey? “Of over 4 million questionnaires sent this year, the magazine received responses regarding about 480,000 vehicles,” according to Detroit News. Now, if most people reported on two cars (because most families have two or more cars), that would put the response rate at a mere 6%. Even assuming one car per family – a highly dubious assumption – we have a taudry 12% response rate.

If you forced them to release this data (which I agree, is the best source for car quality info) then the Service Managers will be told to bury claims…Like they are doing now. Even Lexus buries repair work…NO doubt.

I agree that customer feed-back loops are not 100% reliable/accurate for the simple truism that People Lie. A lot of people base their car purchases solely on pretty designs (still). The car, to them, is a “member of the family” (similar to the people who consider a dog or cat a “human” member of thier family) and not an inanimate tool for transportation. When they encounter problems with the car their ego gets in the way of objective reality. It is like a mother with a newborn child, that has arms protruding from its head and eyes in its chest, will always proclaim “isnt my baby the most perfect baby in the world”. Their are some that will say “that review was totally biased…How can they pick A over B as being a better car”. To those people the “better” car is the subjectively “prettier” one regardless of how reliable it is. Reliablity feed-back loops will continue to evolve toward 100% accuracy though simply because corporate deceit cannot survive for long in a world with worldwide, instantaneous communications (Internet). Decentralized communications will keep/make them honest…Or they will die. But until the feed-back loops are perfected, FOLLOW THE MONEY !!!

You really think Consumer Reports has THAT much juice? Aside from marketing meetings targeting the latte-sipping-and-NPR-listening-all-the-time demographic, no manufacture gives too much weight to Consumer Reports. Well, maybe Toyota…
If a formula contains SUBJECTIVE factors like ride, handling, and aesthetics, it’d be nice to know how different vehicles stack up. (And how brutal inconsistency may reflect on the reviewers’ integrity).

Years ago, “Best selling” book lists from Real Important Newspapers used ’select’ bookstores to compile their data. Then along came Amazon and rude questions were asked about data…

Good luck, TrueDelta.

CR loves Japanese cars, but to say they have a bias against domestics is exaggeration. I bought an Olds Intrigue 5 years ago in part because both CR and Car & Driver raved about it. I found CR’s statement of “average” reliabilty for the Intrigue to be pretty accurate in my ownership of the car.

Mark, you touch on this, but I will expand that thought.
1. They report how good certain vehicles are.
2. Their subscribers (after reading the article)are biased to purchase and enjoy those vehicles.
3. Their subscribers fill out the surveys about their new vehicles and rate them similarly to the way they were rated in the magazine.
4. The magazine repeats to 1 again.

You can see the obvious issue of using a self-selection process that guarantees that respondents highly value (indeed, they PAY for) information that is directly related to the item included on the survey.
Could you pick a more biased sample (besides manufacturer employees) if you wanted to?

CR could use a random sample and would deal with the high expense of doing so. Or perhaps they are concerned with how different the results could be when compared to the present methodology?

My point was simply that for the inquisitive CR reader, pretty much all the information you need is given to you which makes the overall-score/top picks pretty irrelevant.

They do have excellent car reviews in my opinion however.

Btw, you’ve got a great site, it’s a truly useful objective tool in measuring cars on a level plane.
If there’s something I wish it had though, is more rebate information during the pricing comparison.

Anytime I want to see if a car has a rebate, I use this site:

http://www.cars.com/go/advice/incentives/incentivesAll.jsp#Saab

But they don’t list if the rebates are available for MY region. Which makes the list sort of useless. I was hoping your website would have more information on that for a truer price comparison. Still, it’s great stuff though.

@ Luther – Very funny! And, true.

@ Joe – Looking at the mag, the XC90 is probably not recommended due to reliability. It is the lowest-scoring Volvo in reliability.

(sorry for the rambling)

Is CR joking? There is all the proof anyone needs to see that there is bias in CR’s rating system.

GM: “I dont like how you rated our car”
CR: “F-Off”
…Click…

Road&Track:

GM: “I dont like how you rated our car”
RT: “Ohh…um…Yes SIR…um…Ill fix that immediately…SIR”
GM: “Now lets talk about that 4-page color glossy ad we are considering……”

Scores for a couple of the models they reported on were a bit confusing. The Ford Ranger always got more “average” and “below average” circles in the reliability graphs than did the Mazda B-Series truck. Same for the Toyota Corolla compared to the Chevrolet Prism. Now it wasn’t that big of a difference, but there was a difference. Odd……..as these both came off the same assembly line. ( I know this has been mentioned before, but these are apples to apples comparisons ).

This has to be due to perception, or possibly due to a smaller degree, the customer’s denial of the problems that really exist in the car they purchased. For example — the Japanese customer/CR reader may not consider an identical defect or problem in their car as serious as the American customer. And why shouldn’t they? CR has already told them that there car is more reliable, so it must be true.

There also may be a bit of shame at play here. The Corolla customer is going to be a bit more reluctant to report a “serious” problem about their car, IF one of the deciding factors in their purchase was the fact that the car was supposed to be more reliable.

It’s hard to be be truthful when reporting about any problems if that was the basis for buying the car in the first place. And this is CR’s demographic…………..

Their reliability testing…that is a different story (to a certain extent).

Someone also brought up that they didn’t recommend a new Escort but endorsed a used one. That could make sense, especially if a big hit of depreciation up front leaves buyers with a reliable car. I would assume that is the reason, as such a car could still be a good value proposition.