Ask the Best and Brightest: Is There Car Review Star Inflation At TTAC?

ask the best and brightest is there car review star inflation at ttac

A member of the TTAC B&B wrote in to Robert this morning with some unhappy feelings. He feels that stars for cars are going out like free money in the recent months at The Truth About Cars. In this reader’s words:

I had no idea that writers’ star ratings are not edited. Are writers given any instructions on how to rate? In keeping with TTAC’s take no prisoners reputation, I would like to see some uniformity in the ratings where

Top 5% get 5 stars


Next 15% get 4 stars


Middle 60% get 3 stars


Next 15% get 2 stars


Bottom 5% get 1star

In other words, I want to see most cars get a 3-star rating and, as a corollary, a 4-star or 5-star rating to really mean something. This would be really helpful to car shoppers. Currently, however, your writers seem to be giving 4-star ratings out like hot cakes.

Well here’s your chance, B&B. Tell us if you think we are experiencing star inflation. Obviously, our usual rules about flaming the site are suspended. But keep it civil, or I’m going to bust out the whoopin’ stick.

Comments
Join the conversation
4 of 74 comments
  • Dr Lemming Dr Lemming on Sep 11, 2008

    I agree with Phil Ressler -- get rid of the star ratings. TTAC doesn't have the editorial infrastructure to create a ratings scheme that is "empirical" enough to mean something (at least to those with statistical background). I wonder whether you'd want to go that route anyway. Seems to me what you "sell" is context grounded in the subjective but thoughtful opinions of individual reviewers. The star system works against that branding because it implies a comparability across reviews. TTAC reminds me of C&D of yore -- its greatest asset was the personality of its writers. Keep that in the forefront. Star ratings muddy the waters.

  • Jkross22 Jkross22 on Sep 11, 2008

    Thanks for asking. Since there seems to be near universal contempt for grading on a curve, why not grade the way grading was supposed to happen? 0-100%. Interestingly, I've seen video game reviews done this way, and if something cracks 90%, it really is a terrific game. 90-100 - Truly outstanding, class leading or creating a new genre. Aston Martin, Mini, Porsche Cayman, etc. 80-90 - Very good to excellent. Near the top of the class, but slight improvements needed. Audi A6, Corvette, Genesis, Accord 70-80 - Good to very good - first gen cars that need considerable work to be considered class worthy, but a good first try, or cars with history but haven't reached the top of the segment. VW Golf, Passat, G8, etc. 60-70 - Below average to good - Chevy Aveo, Cobalt, etc. 50-60 - Why did the mfg bother to below average - Oh, let's see how many Chrysler products fit here. Hmm. How bout all of em. Less than 50 - You shouldn't be reviewing unless as a joke - Yugo, Hyundai Excel from '89, Suzuki Samari, etc.

  • Ricky Spanish Ricky Spanish on Sep 11, 2008

    I'll be honest - I don't even look at your star system - half the time its inconsistent with the review itself.

  • Akatsuki Akatsuki on Sep 13, 2008

    You guys should make a huge list of cars, and then insert cars where they belong in order. Also make the list dividable by category. As new cars come out, then they can get rated appropriately. The problem with star ratings is a car that is worth 5 stars, can easily be superseded by another car that also gets 5 stars. Or you can just give them all numeric scores behind the scenes and then normalize them to stars on a bell-curve. This means scores will shift as new cars are added, but it also means you get rankings that reflect the state of the market.

Next