By on August 18, 2009

While Automotive News [sub] trumpets the fact that “Cadillac Joins Lexus atop Study of Customer Satisfaction,” our experience with all manner of stat house slickery behooves us to dig a little deeper into the subject. [“league table” here.] While we eagerly await Mr. Karesh’s analytical evisceration, I’d like to share some relevant facts. First, although the “see there IS a perception gap” study is called the American Customer Satisfaction Index™, it hails from the University of Michigan. Second, it’s a racket. The academics behind the index charge companies a $35K “corporate subscription price.” If you don’t work in automotive, no problem! The ACSI covers 44 industries! Oh, and the US government. Third, methodology (as above) . . .

The indexes (shown in the diagram above) are multivariable components measured by several questions that are weighted within the model.  The questions assess customer evaluations of the determinants of each index.  Indexes are reported on a 0 to 100 scale.  The survey and modeling methodology quantifies the strength of the effect of the index on the left to the one to which the arrow points on the right.  These arrows represent “impacts.”  The ACSI model is self-weighting to maximize the explanation of customer satisfaction (ACSI) on customer loyalty.  Looking at the indexes and impacts, users can determine which drivers of satisfaction, if improved, would have the most effect on customer loyalty.

Confused? You should be. Suffice it to say, the man behind the program, Claes Fornell, loves him some Wall Street. Wikipedia: “He is also the lead author of a Journal of Marketing article linking satisfaction performance to stock market growth, demonstrating that a portfolio of firms with high ACSI scores consistently outperforms the market.” That’s got to help his people sell some of them $35K subs, eh?

I wonder if anyone’s ever measured ASCI’s customer satisfaction? I know one guy they should/shouldn’t talk to: Mark Fields. USA Today‘s unquestioning coverage of this domestic heartening stat-fest ends with a quote from Ford’s el presidente de las Americas.

I think in our case, the consumers are seeing new products, they’re seeing information from third parties on our quality, and it’s all raising awareness,” he says. “All of this highlighting in the media every day is helpful to us, given that we have the evidence to back up our claims.”

Define evidence.

Get the latest TTAC e-Newsletter!

17 Comments on “Another Day, Another Dubious Satisfaction Survey Lauding Detroit...”


  • avatar
    Boff

    Racket? Universities charge the private sector for products, services, and IP all the time. Is Wolverines football a racket?

    I invent a compound to cure cancer. I get my university’s technology transfer office to arrange a patent. I publish papers extolling the efficacy of the drug. I speak at international conferences. (Making the appropriate disclosures at all times, of course). University licenses IP to big drug company and rakes in $$$. What’s the problem here?

    If the drug or survey methodology turns out to not be efficacious, the venture fails.

  • avatar
    carguy

    Boff – this ain’t no cure for cancer. It doesn’t produce anything except distortion designed to benefit those that pay the subscription fee. Universities are supposed to create knowledge, not trade on their academic credibility in order to push market distortions for profit.

    dkulmacz – I don’t think they care who they do it for, domestic or otherwise. Maybe they had existing contacts with the domestic car makers and that is why they are their first customers. I’m sure if Toyota paid more they would change their minds…

  • avatar
    Kristjan Ambroz

    Universities often need funds and paid projects in a consulting capacity, which this is one off, are quite common. In theory these offer several benefits – the industry learns about new approaches from academia, academia learns what pertinent issues in the industry are and there’s scope for plenty of PhDs built in.

    Unfortunately, especially in the management field the understanding is not sufficiently strong to provide a good product to the client, or, alternatively, as often in consulting, the client cannot assess the quality of the deliverable, universities offer a cheaper service than a reputable consultancy and gets hired for the purpose. I am not tryingto assert universities / departments knowlingly engage in wrongdoing but sometimes professors are wedded to ‘beautiful theories and do not like it, when ugly facts slaughter them.’

  • avatar
    rnc

    Define evidence.

    Consumer reports, Edmunds (and all of the smaller quality surverys that are disregarded), increasing market share especially when considered in relation to selling price. A majority of the B&B (if you read the comments on editorials about Ford they’ve become by in large positive) Would those qualify as evidence?

  • avatar
    slateslate

    Naturally surveys in general have their flaws but that doesn’t preclude the fact that Caddy/Buick/Lincoln owners love their cars.

    Just as Porsche owners still love their cars after an engine failure or annual bushing replacements, a DTS/Lucerne owner probably loves their car for all the qualities that drew them to that model in the first place as long as there is no major quality issue with the car.

    The worst part of surveys like this is that management will use this survey to say, “Hah, I told you so….there is a perception gap.”

    The big problem is that the average Lincoln/Buick owner is literally dying off before a conquest buyer replaces them.

  • avatar
    WetWilly

    I believe Mr. Farago’s point is that the survey results are themselves a product and the normal biases of product development, i.e. giving customers what they want, will affect the product. After all, who’d pay $35K/subscription based on a study that says Lexus and Toyota are tied at #1 for customer satisfaction?

  • avatar
    derm81

    UM is one of the best schools in the country. They could be in research to make a buck, but when UM comes out with a study, it is going to be spot on. UM created the University of Michigan Consumer Sentiment Index which is one of the finest tools used by economists today. I trust UM’s data more than data and studies from a private company any day of the week.

  • avatar
    Gardiner Westbound

    Clients invariably receive the desired survey results.

  • avatar

    I just received a JD Powers survey for my recently purchased (new) Volvo C30 via snail mail.

    How much would it be worth to my local Volvo dealer if I sold them the survey link to avoid a truthful review of car and service?

    More accurately how much would it be worth to the dealer and to Volvo to avoid a truthful telling of the story of my four out of round tires that were replaced with four brand new tires, which were then promptly destroyed because they never bothered to check the alignment?

  • avatar
    ryv

    Ok, after reading some of the comments I thought I’d chime in for once here. Why pay $35k for the ACSI index? Because it allows companies to see not only where (in what areas) they are failing to win over customers but in what ways their competitors are, which eventually allow them to differentiate themselves or give them a high road to take when new advertising times come around. Additionally, $35k is cheap in Market Research world – what do you think JD.Power and CO charges for their Initial Quality Study?

    There’s no such thing as being able to push all error or bias out of market research studies – a smart company looks at the methodology and judges for themselves that a proper sample is collected and that the metrics analyzed are sound. At the end of the day though when a market research study tells you something you didn’t expect, its incredibly valuable. Just because its findings doesn’t sync with what you thought doesn’t mean the findings are wrong… maybe, just maybe it means you are?

    That said looking at that crazy ACSI flowchart, they have their own problems with weights but I would say the idea of the output from that is to predict your customer complaints before they happen based on how consumers are responding to your products. And thats pretty valuable if your trying to maintain a corporate image.

  • avatar
    texlovera

    Sorry, but as an engineer, when I see verbage such as “…quantifies the strength of the effect of the index…”, my bullshit detector hits at least a 6 or 7. And that is based on a lotta years of seeing bullshit consultant work.

  • avatar
    cdotson

    texlovera: agreed, although I do understand that were this a factorial DOE that phrase may be trying to state which index had the stronger affect on outcome…

    But while repeatedly using “indexes” instead of “indices” might be acceptable in the new school it might be indicative of someone using New Math as well.

  • avatar
    jerry weber

    the name jd powers came up in the blog. It is not that powers is wrong it is that they are the quitessinal marketing machine. There are so many categories for cars to rate in it is mind numbing. There is the initial quality, after 90 days etc. Let’s see why they need 99 different scales to rate cars and trucks. If you keep slicing the apple from another direction a new winner keeps appearing. Thus, almost all of the auto companies have a winner somewhere in this lottery. Everyone has a trophy to picture on their new products representing a win in some category. They do it with colleges. Your child can go to a winning college anywhere. They all won in some category and this they tout as the reason you should use them. Is it meaningful? You tell me.

  • avatar
    Detroit Todd

    First…it hails from the University of Michigan

    Are you insinuating that a thumb was placed on the scales in favor of domestics because the survey emanated from the University of Michigan? I don’t think so. Maybe U of M performed the study because all manner of high-level automotive expertise — engineering, manufacturing, marketing and otherwise — is located in Michigan.

    The University of Michigan hails from Ann Arbor. Guess what else does? Toyota’s recently expanded Technical Center and soon-to-be-built North American Research & Development Center.

    While we eagerly await Mr. Karesh’s analytical evisceration

    Without even hearing from Mr. Karesh, you assume he’s going to tear this survey to shreds, Hulk-style? How objective of you!

    That a university — a world renowned one — reaches a conclusion you do not like automatically means the study is crap?

    Pardon me, but I (and I’m sure nearly everyone else) will not dismiss the conclusions of The University of Michigan simply because you are encountering cognitive dissonance.

    “He is also the lead author of a Journal of Marketing article linking satisfaction performance to stock market growth, demonstrating that a portfolio of firms with high ACSI scores consistently outperforms the market.” That’s got to help his people sell some of them $35K subs, eh?

    Since he was successful in the private sector, the guy has no business working on a study under the auspices of a prestigious university? What? I don’t see anything contentious about this.

    Further, to leverage U of M’s intellectual heft to do anything is going to cost you some money. Nothing nefarious about that, either.

    Define evidence.

    Indeed. It is up to you (or Mr. Karesh, or Ron Paul, Ru Paul, or whomever you choose to speak on your behalf) to prove the survey is flawed and/or biased. Being cranky about the survey results because they hurt your feelings doesn’t count.

    I do not think you will be successful.

    In fact, you may want to consult a U of M surgeon to remove that giant chip you have on your shoulder with respect to the domestic automotive industry.

  • avatar
    RogerB34

    A successful enterprise based on circular logic.

  • avatar
    MichaelJ

    Detroit Todd:

    Without even hearing from Mr. Karesh, you assume he’s going to tear this survey to shreds, Hulk-style?

    Of course the expectation is that Mr. Karesh will tear the survey to shreds. After all, UofM must be biased to the domestics simply because of their location, right?

    Therefore Mr. Karesh must be biased to this site and will agree with the prevailing preconception here that everything domestic is bad…

    “Another day, another dubious satisfaction survey lauding Detroit.” Implies these are coming in daily. How many such dubiousnessosities does it take before maybe…maybe…possibly…there’s a little glint of acceptance that they’re not actually wrong?

    I sure wish we could start seeing the honorable truthful non-biased scientific surveys that agree with the sentiment here that Detroit=crap and imports are perfect.

  • avatar
    Happy_Endings

    It all goes back to the old Onion headline; 98% of statistics are made up on the spot.

Read all comments

Back to TopLeave a Reply

You must be logged in to post a comment.

Recent Comments

  • Lou_BC: @Jeff S – I’ve moved between truck classes based on my needs. My 1st truck was a regular cab...
  • Corey Lewis: Wow, dark end.
  • Oberkanone: Nice vehicle. It’s outdated goodness is reminder of Toyota quality of the past when vehicles were a...
  • bullnuke: My neighbor worked for Frigidaire in Moraine. When it shutdown he came home after stopping by the bar up on...
  • FreedMike: Oooooh, stripes! Snark aside, the look is actually pretty cool. Toyota seems to have gotten a good deal on...

New Car Research

Get a Free Dealer Quote

Who We Are

  • Adam Tonge
  • Bozi Tatarevic
  • Corey Lewis
  • Jo Borras
  • Mark Baruth
  • Ronnie Schreiber