U.S. News Clarifies Their Auto Ranking Process. Or Not.
U.S.News & World Report believes in editorial transparency. So they've published the methodology they use to rate virtually every new car sold in America. As you'd expect from such a soberly-titled title, the recommendations are based on "hard data and analysis." First, they convert reviews by unnamed "respected auto critics" into a numeric score, using an undisclosed process. Then, they add information "such as safety and reliability data" that's scored and weighted according to an unspecified formula. After that, they combine all the scores and calculate them using a process "based on [unrevealed factors that] consumers say matters to them most in a car." The result? An overall score that allows their editors to compare cars, "ranking them against each other." They do admit that sometimes "the overall score is higher or lower than the component scores we display" because they also consider the "author's level of recommendation or overall impression about the car." OK, to sum up, they arbitrarily assign scores based on their assessment of someone else's assessment, add in their ranking of someone else's safety and reliability data, and out pops a number that may or may not be the final score, based on their author's arbitrary opinion of the vehicle. And that's the truth. [thanks to starlightmica for the link]
How hard is it really to make a ranking of automobiles? You don't need any set formulas at all. Just follow the common method that all automotive magazines use to make a "Best of" list: 1. include at least 1 BMW for "driving dynamics" 2. include at least 1 American car as a "suprise quality hit that shows corporate culture is changing" 3. Prius 4. 911/Cayman/Boxster 5. Corvette 6. include an Infiniti/Acura as the "smart alternative to the Germans" 7. High MPG non-hybrid. 8. Miata or the MazdaSpeed3 Find a list that doesn't have this stuff included.
US News specializes in catering to readers' desire for expert opinions on practical stuff (news you can use, in their parlance). Even people who aren't in the market scan this stuff avidly to see if what they have is "the best." I'm going to be coming out with my list of "best bridges to buy for $200." I've already determined that Boston's (my town) Leonard Zakim Bunker Hill Bridge will be the winner, followed by the Brooklyn Bridge, but please, as a courtesy to a fellow car enthusiast, don't tell the people at US News about this.
Frank, hilarious piece, thanks. At least they didn't add "statistical analysis" to their assessments!
Isn't that kinda of how Google ranks web pages? Well, at least it isn't as bad as most magazines who give top rankings to their largest advertisers.