The Truth About J.D. Power's IQS

Michael Karesh
by Michael Karesh

Another year, another J.D. Power survey. Since the non-profit Consumer Reports organization prohibits carmakers from using its ratings in their ads, “ranked highest in initial quality by J.D. Power and Associates” should start flooding the airwaves and Internet any minute now, with print sure to follow. But does all of this noise signify anything? Should those seeking trouble-free wheels be sure to buy one of J.D.’s winners? Hardly.

First, note the “initial” that qualifies “quality.” Power surveys car owners on “problems” encountered within their first 90 days of ownership. Most people understand that a car that’s reliable for 90 days isn’t necessarily reliable beyond that. But there’s a bigger issue. J.D. Power’s IQS has been redesigned (for the second time) to encompass a larger number of potential defects. And the more the IQS includes, the less it measures what most people want to know: vehicle reliability.

The previous redesign doubled the average number of reported problems per car by extending the IQS beyond defects (that can be fixed) to designed-in annoyances (that must be endured). For example, cupholder dissatisfaction famously slammed MINI’s score. The 2006 IQS report takes a step in the right direction by including subscores for "design quality" and "production quality." Combining two very different elements into a single score makes it unclear what the number represents. Yet this score receives 99 percent of the press coverage and 100 percent of the ad citations.

If you compare the rankings based on production quality alone, the brands’ relative positions change dramatically. BMW bounds 24 places to third; Buick jumps 14 to eighth; MINI ascends 13 to 16th; Mercedes-Benz climbs nine also to 16th; Subaru also gains nine to 19th. At the same time, Dodge drops eight to 27th; GMC plummets 13 rungs to 22nd; Nissan plunges ten, also to 22nd. Eight others change position by at least five slots. These include Chrysler, which shares many models with Dodge yet moves up five places, to fifth. Out of 37 brands, 16 rankings are heavily affected by the inclusion of design quality.

Beyond the cloudiness of the revised methodology, the way the results are reported and spun continues to put too much emphasis on relative rankings. In fact, absolute differences are often minuscule. Looking at defect rates alone, 22 out of 37 brands fall within one-tenth of a Problem per Car (PPC) of the 0.64 average. Thirty of 37 brands fall within two-tenths. Of the seven beyond this range, only one, Lexus, is on the top, and it only betters the average by 0.22 problems per car.

Stay with me here. The best brand, Lexus, has 0.42 problems per car, while the worst, Isuzu, has 1.10: a best-to-worst difference of 0.68 problems per car. Even this range results from a few especially low-scoring brands. The difference between number three (Toyota) and number 32 (Hummer) is a scant 0.27 problems per car. It’s ironic, since brands at the bottom of the chart receive the least attention in J.D. Power’s press releases. For years they didn’t even publicly release below-average scores.

Put another way, a Toyota compared to a Hummer has a one in four chance of having a single additional problem. Even comparing a car from Isuzu with one from Lexus, only two in three cars will have a single additional problem. What’s more, this additional problem is likely to be the only problem. Folks, we're talking about a single trip to the dealer for a single problem–which you still face nearly even odds of taking if you buy the best brand.

The reason why J.D. Power lumps design quality into the IQS is clear: without it, the differences between brands are rarely worth debating. And the smaller the differences, the less people care about IQS. And the less the pubic cares about IQS, the less automakers will pay to advertise IQS scores, and hire Power consultants to help improve them. This would truly be a problem– for J.D. Power.

J.D. needs to re-think their methodology and reporting. They should keep problems that require repair separate from other issues. Forget the brands, they don’t vary enough. Instead, emphasize model scores. Next, focus less on rankings and who is the best and more on the size of the differences and who is within spitting distance of the best. Finally, J.D. Power needs to shift their emphasis away from “initial quality” towards long-term durability. Manufacturers won’t like that a longer-term study keeps new models off J.D.-branded consumer radar, but anything less is, well, less.

Heck, J.D. Power might even rake in more cash this way. Anyone reasonably near the top—and not just those at the top—could advertise “ranked good enough in quality that you should focus on other criteria by J.D. Power and Associates.” No, it’s not punchy. Just the truth.

[Michael Karesh operates www.truedelta.com, a vehicle reliability and price comparison site.]

Michael Karesh
Michael Karesh

Michael Karesh lives in West Bloomfield, Michigan, with his wife and three children. In 2003 he received a Ph.D. from the University of Chicago. While in Chicago he worked at the National Opinion Research Center, a leader in the field of survey research. For his doctoral thesis, he spent a year-and-a-half inside an automaker studying how and how well it understood consumers when developing new products. While pursuing the degree he taught consumer behavior and product development at Oakland University. Since 1999, he has contributed auto reviews to Epinions, where he is currently one of two people in charge of the autos section. Since earning the degree he has continued to care for his children (school, gymnastics, tae-kwan-do...) and write reviews for Epinions and, more recently, The Truth About Cars while developing TrueDelta, a vehicle reliability and price comparison site.

More by Michael Karesh

Comments
Join the conversation
2 of 3 comments
  • Michael Karesh Michael Karesh on Jun 15, 2006

    It looks like the Strategic Vision has succeeded where social scientists have repeatedly failed: scientific proof for Maslow's hierarchy of needs (which should be familiar to anyone who has had Marketing 101). http://en.wikipedia.org/wiki/Maslow's_hierarchy_of_needs This sort of theory of values is old stuff. My M.A. paper concerned this sort of research back in the 1950s and 1960s. Connecting such values to purchasing behavrior has always been iffy at best. Tends to fall apart like wet toilet paper when pressed. People just refuse to do what they're expected to do when it's time to buy a car. That said, Strategic Vision does try harder than JD Power to explore and capture the nuances of automotive satisfaction. It's hard to say how well they succeed at this without more details than they publicly release.

  • Robert Farago Robert Farago on Jun 15, 2006

    Yes, they are a bit reluctant to release their precise methodology. Commerical secret or too wild and wooly for public consumption? GM quotes their results an awful lot, and they're customers. But then all of these JD Power-type guys see no problem in accepting money from the people about whom they report. Who's to say the Chinese Walls don't have ears? OK, gotta pee.

  • Steve Biro Frankly, while I can do without Eyesight and automatic start-stop, there is generally less B-S with Subarus in terms of design, utility and off-road chops than with many other brands. I just hope that when they adopt Toyota’s hybrid system, they’ll also use Toyota’s eCVT.
  • The Oracle These are all over the roads in droves here in WNC. Rarely see one on the side of the road, they are wildly popular, capable, and reliable. There is a market for utilitarian vehicles.
  • Stephen My "mid-level" limited edition Tonino Lambo Ferraccio Junior watch has performed flawlessly with attractive understated style for nearly 20 years. Their cars are not so much to my taste-- my Acura NSX is just fine. Not sure why you have such condescension towards these excellent timepieces. They are attractive without unnecessary flamboyance, keep perfect time and are extremely reliable. They are also very reasonably priced.
  • Dana You don’t need park, you set auto hold (button on the console). Every BMW answers to ‘Hey, BMW’, but you can set your own personal wake word in iDrive. It takes less than 5 minutes to figure that that out, btw. The audio stays on which is handy for Teams meetings. Once your phone is out of range, the audio is stopped on the car. You can always press down on the audio volume wheel which will mute it, if it bothers you. I found all the controls very intuitive.
  • ToolGuy Not sure if I've ever said this, or if you were listening:• Learn to drive, people.Also, learn which vehicles to take home with you and which ones to walk away from. You are an adult now, think for yourself. (Those ads are lying to you. Your friendly neighborhood automotive dealer, also lying to you. Politicians? Lying to you. Oh yeah, learn how to vote lol.)Addendum for the weak-minded who think I am advocating some 'driver training' program: Learning is not something you do in school once for all time. Learning how to drive is not something that someone does for you. It is a continuous process driven by YOU. Learn how to learn how to drive, and learn to drive. Keep on learning how to drive. (You -- over there -- especially you, you kind of suck at driving. LOL.)Example: Do you know where your tires are? When you are 4 hours into a 6 hour interstate journey and change lanes, do you run over the raised center line retroreflective bumpers, or do you steer between them?
Next