Ask the Best and Brightest: Is There Car Review Star Inflation At TTAC?
A member of the TTAC B&B wrote in to Robert this morning with some unhappy feelings. He feels that stars for cars are going out like free money in the recent months at The Truth About Cars. In this reader’s words:
I had no idea that writers’ star ratings are not edited. Are writers given any instructions on how to rate? In keeping with TTAC’s take no prisoners reputation, I would like to see some uniformity in the ratings where
Top 5% get 5 stars
In other words, I want to see most cars get a 3-star rating and, as a corollary, a 4-star or 5-star rating to really mean something. This would be really helpful to car shoppers. Currently, however, your writers seem to be giving 4-star ratings out like hot cakes.
Well here’s your chance, B&B. Tell us if you think we are experiencing star inflation. Obviously, our usual rules about flaming the site are suspended. But keep it civil, or I’m going to bust out the whoopin’ stick.
Immensely bored law student. I've also got 3 dogs.
More by Justin Berkowitz
Comments
Join the conversation
I agree with Phil Ressler -- get rid of the star ratings. TTAC doesn't have the editorial infrastructure to create a ratings scheme that is "empirical" enough to mean something (at least to those with statistical background). I wonder whether you'd want to go that route anyway. Seems to me what you "sell" is context grounded in the subjective but thoughtful opinions of individual reviewers. The star system works against that branding because it implies a comparability across reviews. TTAC reminds me of C&D of yore -- its greatest asset was the personality of its writers. Keep that in the forefront. Star ratings muddy the waters.
Thanks for asking. Since there seems to be near universal contempt for grading on a curve, why not grade the way grading was supposed to happen? 0-100%. Interestingly, I've seen video game reviews done this way, and if something cracks 90%, it really is a terrific game. 90-100 - Truly outstanding, class leading or creating a new genre. Aston Martin, Mini, Porsche Cayman, etc. 80-90 - Very good to excellent. Near the top of the class, but slight improvements needed. Audi A6, Corvette, Genesis, Accord 70-80 - Good to very good - first gen cars that need considerable work to be considered class worthy, but a good first try, or cars with history but haven't reached the top of the segment. VW Golf, Passat, G8, etc. 60-70 - Below average to good - Chevy Aveo, Cobalt, etc. 50-60 - Why did the mfg bother to below average - Oh, let's see how many Chrysler products fit here. Hmm. How bout all of em. Less than 50 - You shouldn't be reviewing unless as a joke - Yugo, Hyundai Excel from '89, Suzuki Samari, etc.
I'll be honest - I don't even look at your star system - half the time its inconsistent with the review itself.
You guys should make a huge list of cars, and then insert cars where they belong in order. Also make the list dividable by category. As new cars come out, then they can get rated appropriately. The problem with star ratings is a car that is worth 5 stars, can easily be superseded by another car that also gets 5 stars. Or you can just give them all numeric scores behind the scenes and then normalize them to stars on a bell-curve. This means scores will shift as new cars are added, but it also means you get rankings that reflect the state of the market.