Grade The Analysts: Jessica Caldwell Crowned Most Reliable Analyst Of 2012

Bertel Schmitt
by Bertel Schmitt
grade the analysts jessica caldwell crowned most reliable analyst of 2012

If your life, career, wealth, or all of the above depend on forecasting the auto market with precision and reliability, then you can do no better than securing the services of Edmunds’ chief analyst Jessica Caldwell. Last year’s winner of the coveted “TTAC 11” (a.k.a. “The Top Analyst Crown 2011”) does it again: Jessica Caldwell wins the TTAC 12, closely followed by her perennial antagonists, Jesse Toprak of TrueCar and Peter Nesvold of Jefferies.

TTAC congratulates Jessica Caldwell and her team to her second coronation in a row.

TTAC Analyst Ranking 2012, Final StandingsRankAnalystAverage Deviation1Jessica Caldwell ( Toprak ( Nesvold (Jefferies)17.08%4Alec Gutierrez (Kelley)18.08%5Brian Johnson (Barclays)24.60%6Patrick Archambault (Goldman)35.40%7Joseph Spak (RBC)40.45%8Rod Lache (Deutsche Bank)48.10%9Chris Ceraso (Credit Suisse)49.43%10Emmanuel Rosner (CLSA)80.80%11John Sousanis (Ward’s)110.95%12Alan Baum (Baum & Associates)302.64%13Jeff Schuster (LMC Automotive)302.85%14George Magliano (IHS)311.23%15Adam Jonas (Morgan Stanley)342.89%16Itay Michaeli (Citigroup)359.90%17Matthew Stover (Guggenheim)383.83%18Himanshu Patel (JPMorgan)384.23%19Christopher Hopson (IHS)391.83%

Each month, Bloomberg asks a panel of America’s best and (hopefully) highest paid analysts how this month’s auto sales will come in. Each month, TTAC grades the analysts on how well they hit their target. Each month, some of our readers ask why.

The auto business lives on reliable forecasts. Marketers and engineers have to gauge customer desires many years ahead. Budgets have to be made, resources allocated, production schedules have to be put together, parts must be ordered, advertising dollars allocated, cars shipped to dealer lots. What’s more, car sales are an indicator for the economy. That’s why brokerage houses and banks employ hordes of analysts, that’s why specialist firms like Edmunds, TrueCar, Kelley, IHS and LMC invest considerable energy into forecasting the market. And that’s why TTAC benchmarks analyst results.

Nobody can predict the future with 100% accuracy. Panels of reporting dealers can be built that bring you pretty close to the actual results. Fleet sales make the matter tricky. A volume deal, moved a day back or ahead can throw off the best forecast.

When you look for a good forecaster, you look for reliability and consistency. The hero is not the forecaster who hits a bulls-eye once. A good forecaster is close to the truth most of the time. Jessica Caldwell is a great forecaster. A good forecaster never takes a vacation, he or she never shrugs the shoulders and says “I don’t know.”

If I would have to recommend the perfect forecaster, then I would name a panel consisting of Jessica Caldwell, Jesse Toprak of Truecar, and Peter Nesvold of Jefferies. Between them, they have six first places last year, eight second places and five third places. Actually, I would have a chat with Brian Johnson of Barclays and put him on the panel if he promises never to punt again. In April last year, Brian shrugged his shoulders when it came to Chrysler and said “I don’t know.” This slip-up did cost Brian a podium finish in the annual TTAC ranking. Did we mention consistency and reliability?

TTAC Analyst Rankings December 2012RankAnalyst GMFord Chrysler SAARSAAR DiffOEM DiffOverall1Brian Johnson (Barclays)6.7%0.6%10.0%15.50.8%3.1%3.9%2John Sousanis (Ward’s)4.0%1.0%9.0%15.72.1%3.0%5.1%3Jessica Caldwell ( Ceraso (Credit Suisse)2.0%3.1%8.1%15.50.8%6.0%6.8%5Jesse Toprak ( Spak (RBC)2.6%0.8%7.1%15.61.5%6.5%8.0%7Rod Lache (Deutsche Bank)3.0%-2.0%9.0%15.11.8%7.0%8.8%8Peter Nesvold (Jefferies)0.0%1.2%7.3%15.30.5%8.5%9.0%9Patrick Archambault (Goldman)1.4%3.0%6.4%15.50.8%8.2%9.0%10Emmanuel Rosner (CLSA)0.3%0.7%5.1%15.40.2%10.9%11.1%11Alec Gutierrez (Kelley)0.4%0.3%5.9%15.21.1%10.4%11.5%12Ryan Brinkman (JPMorgan)NANANA15.40.2%300.0%300.2%13Alan Baum (Baum & Associates)NANANA15.40.2%300.0%300.2%14Jeff Schuster (LMC Automotive)NANANA15.30.5%300.0%300.5%15George Magliano (IHS)NANANA15.21.1%300.0%301.1%Average2.1%1.2%7.6%15.40Actual5.0%2.0%10.0%15.37

Case in point: In December, Brian Johnson took the top post, with Caldwell in place three, Toprak in 5 and Nesvold in eight. Brian took two first last year, one second and two thirds. Would he not have chickened out on Chrysler last April, Johnson probably would have a podium finish in the annual rankings in addition to his first place in December.

How does one predict sales with accuracy, reliability and consistency? Apart from the obvious (guess, call the OEMs, establish a real time dealer panel, knock on wood) the intricate details remain a closely guarded secret. Asked how in the world she can be so good, Jessica Caldwell delivered a politically correct statement that nonetheless won’t help you break into the forecasting business:

“Forecasting is, of course, a team effort. Out team includes several PhDs, statisticians and hundreds of other colleagues who are working to translate this data and deliver key insights to help both shoppers and dealers improve the car buying experience. You won’t find a more hard working group of data experts anywhere else in the industry.”

A word to the also-rans, beginning halfway down the annual list: Only betting on a SAAR number and withholding an estimate for the Detroit Three is a surefire way of ruining your chances of winning. This has sunk in. In January, six out of 14 analysts only provided a SAAR. In December, this number was down to four out of 15. Likewise, analysts who pop up only once or twice in the year stand no chance. It would be better to bet in the name of the vacationing colleague. This is a highly competitive and unforgiving business.

A spreadsheet with the monthly results and other nifty stats can be downloaded here.

Join the conversation
2 of 14 comments
  • AoLetsGo AoLetsGo on Jan 04, 2013

    First Congrats to Jessica and her team. Myself and others at my company do some sales forecasting for clients and we consider anything within +/- 15% a very good score. If I say so myself our job is tougher since we get one shot at a sales number (no monthly revisions) and it can be a year or two out in the future. That said we tell people that while we strive to be 100% accurate what is really important is that we are directionally correct and provide good guidance on go or no go decisions.

  • GiddyHitch GiddyHitch on Jan 04, 2013

    Jessica and her team obviously do great work but I am now wondering how much of her forecasts are driven directly by data gathered from the Edmunds website itself by consumers shopping for a new vehicle. This would be an inside line on potential sales unavailable to other analysts.

  • Ajla $24K electric Beetle plz.
  • SCE to AUX No mention of price, but my guess of $60k from a few years ago may be low.My EPA estimate would be 263 miles, but that's unladen, temperate conditions, driven at the speed limit, and 0% left in the tank - all unrealistic.Subtract 15% for full payload, 20% for cold, 10% for speed, and 20% minimum battery level, and you're down to 129 usable miles at times. Even in nice conditions (springtime, town driving), I'd only expect 180 usable miles.This vehicle will have the same challenge as electric pickups do - when used as intended (traveling with family and stuff in this case), the utility is lost.When these hit US roads, expect to see videos of unhappy/surprised customers who thought this thing would go 260+ miles all the time. For starters, it should have a 150 kWh battery, minimum.
  • Lorenzo They should have put a more upright windshield on it, and kept the V trim on the hood. Dump the sliding passenger door and put the twin hinged doors back. Dump the batteries and put the 1.4 turbo into it, and put the original 16 inch wheels back, with metal hubcaps. VW did to the Buzz what they did to the new Bug, went with the generic shape and took out all the details people liked.
  • Azfelix This inspires so much confidence in the knowledge that government employees also have oversight over the proposed emergency braking rules. Ancient Athenians utilized the process of banishment. Perhaps we should consider implementing it for every government agency at every level.
  • MrIcky Its going to sell really well for a little bit, then everyone who wanted one will have one and it will sell almost nothing ever again-primarily well to do flower shop delivery vehicles after that first wave.