Grade The Analysts: Jessica Caldwell Crowned Most Reliable Analyst Of 2012

Bertel Schmitt
by Bertel Schmitt

If your life, career, wealth, or all of the above depend on forecasting the auto market with precision and reliability, then you can do no better than securing the services of Edmunds’ chief analyst Jessica Caldwell. Last year’s winner of the coveted “TTAC 11” (a.k.a. “The Top Analyst Crown 2011”) does it again: Jessica Caldwell wins the TTAC 12, closely followed by her perennial antagonists, Jesse Toprak of TrueCar and Peter Nesvold of Jefferies.

TTAC congratulates Jessica Caldwell and her team to her second coronation in a row.

TTAC Analyst Ranking 2012, Final StandingsRankAnalystAverage Deviation1Jessica Caldwell (Edmunds.com)12.12%2Jesse Toprak (TrueCar.com)15.55%3Peter Nesvold (Jefferies)17.08%4Alec Gutierrez (Kelley)18.08%5Brian Johnson (Barclays)24.60%6Patrick Archambault (Goldman)35.40%7Joseph Spak (RBC)40.45%8Rod Lache (Deutsche Bank)48.10%9Chris Ceraso (Credit Suisse)49.43%10Emmanuel Rosner (CLSA)80.80%11John Sousanis (Ward’s)110.95%12Alan Baum (Baum & Associates)302.64%13Jeff Schuster (LMC Automotive)302.85%14George Magliano (IHS)311.23%15Adam Jonas (Morgan Stanley)342.89%16Itay Michaeli (Citigroup)359.90%17Matthew Stover (Guggenheim)383.83%18Himanshu Patel (JPMorgan)384.23%19Christopher Hopson (IHS)391.83%

Each month, Bloomberg asks a panel of America’s best and (hopefully) highest paid analysts how this month’s auto sales will come in. Each month, TTAC grades the analysts on how well they hit their target. Each month, some of our readers ask why.

The auto business lives on reliable forecasts. Marketers and engineers have to gauge customer desires many years ahead. Budgets have to be made, resources allocated, production schedules have to be put together, parts must be ordered, advertising dollars allocated, cars shipped to dealer lots. What’s more, car sales are an indicator for the economy. That’s why brokerage houses and banks employ hordes of analysts, that’s why specialist firms like Edmunds, TrueCar, Kelley, IHS and LMC invest considerable energy into forecasting the market. And that’s why TTAC benchmarks analyst results.


Nobody can predict the future with 100% accuracy. Panels of reporting dealers can be built that bring you pretty close to the actual results. Fleet sales make the matter tricky. A volume deal, moved a day back or ahead can throw off the best forecast.


When you look for a good forecaster, you look for reliability and consistency. The hero is not the forecaster who hits a bulls-eye once. A good forecaster is close to the truth most of the time. Jessica Caldwell is a great forecaster. A good forecaster never takes a vacation, he or she never shrugs the shoulders and says “I don’t know.”

If I would have to recommend the perfect forecaster, then I would name a panel consisting of Jessica Caldwell, Jesse Toprak of Truecar, and Peter Nesvold of Jefferies. Between them, they have six first places last year, eight second places and five third places. Actually, I would have a chat with Brian Johnson of Barclays and put him on the panel if he promises never to punt again. In April last year, Brian shrugged his shoulders when it came to Chrysler and said “I don’t know.” This slip-up did cost Brian a podium finish in the annual TTAC ranking. Did we mention consistency and reliability?

TTAC Analyst Rankings December 2012RankAnalyst GMFord Chrysler SAARSAAR DiffOEM DiffOverall1Brian Johnson (Barclays)6.7%0.6%10.0%15.50.8%3.1%3.9%2John Sousanis (Ward’s)4.0%1.0%9.0%15.72.1%3.0%5.1%3Jessica Caldwell (Edmunds.com)1.5%1.6%7.3%15.40.2%6.6%6.8%4Chris Ceraso (Credit Suisse)2.0%3.1%8.1%15.50.8%6.0%6.8%5Jesse Toprak (TrueCar.com)0.9%2.6%8.8%15.61.5%5.9%7.4%6Joseph Spak (RBC)2.6%0.8%7.1%15.61.5%6.5%8.0%7Rod Lache (Deutsche Bank)3.0%-2.0%9.0%15.11.8%7.0%8.8%8Peter Nesvold (Jefferies)0.0%1.2%7.3%15.30.5%8.5%9.0%9Patrick Archambault (Goldman)1.4%3.0%6.4%15.50.8%8.2%9.0%10Emmanuel Rosner (CLSA)0.3%0.7%5.1%15.40.2%10.9%11.1%11Alec Gutierrez (Kelley)0.4%0.3%5.9%15.21.1%10.4%11.5%12Ryan Brinkman (JPMorgan)NANANA15.40.2%300.0%300.2%13Alan Baum (Baum & Associates)NANANA15.40.2%300.0%300.2%14Jeff Schuster (LMC Automotive)NANANA15.30.5%300.0%300.5%15George Magliano (IHS)NANANA15.21.1%300.0%301.1%Average2.1%1.2%7.6%15.40Actual5.0%2.0%10.0%15.37

Case in point: In December, Brian Johnson took the top post, with Caldwell in place three, Toprak in 5 and Nesvold in eight. Brian took two first last year, one second and two thirds. Would he not have chickened out on Chrysler last April, Johnson probably would have a podium finish in the annual rankings in addition to his first place in December.

How does one predict sales with accuracy, reliability and consistency? Apart from the obvious (guess, call the OEMs, establish a real time dealer panel, knock on wood) the intricate details remain a closely guarded secret. Asked how in the world she can be so good, Jessica Caldwell delivered a politically correct statement that nonetheless won’t help you break into the forecasting business:

“Forecasting is, of course, a team effort. Out team includes several PhDs, statisticians and hundreds of other colleagues who are working to translate this data and deliver key insights to help both shoppers and dealers improve the car buying experience. You won’t find a more hard working group of data experts anywhere else in the industry.”

A word to the also-rans, beginning halfway down the annual list: Only betting on a SAAR number and withholding an estimate for the Detroit Three is a surefire way of ruining your chances of winning. This has sunk in. In January, six out of 14 analysts only provided a SAAR. In December, this number was down to four out of 15. Likewise, analysts who pop up only once or twice in the year stand no chance. It would be better to bet in the name of the vacationing colleague. This is a highly competitive and unforgiving business.

A spreadsheet with the monthly results and other nifty stats can be downloaded here.

Bertel Schmitt
Bertel Schmitt

Bertel Schmitt comes back to journalism after taking a 35 year break in advertising and marketing. He ran and owned advertising agencies in Duesseldorf, Germany, and New York City. Volkswagen A.G. was Bertel's most important corporate account. Schmitt's advertising and marketing career touched many corners of the industry with a special focus on automotive products and services. Since 2004, he lives in Japan and China with his wife <a href="http://www.tomokoandbertel.com"> Tomoko </a>. Bertel Schmitt is a founding board member of the <a href="http://www.offshoresuperseries.com"> Offshore Super Series </a>, an American offshore powerboat racing organization. He is co-owner of the racing team Typhoon.

More by Bertel Schmitt

Comments
Join the conversation
2 of 14 comments
  • AoLetsGo AoLetsGo on Jan 04, 2013

    First Congrats to Jessica and her team. Myself and others at my company do some sales forecasting for clients and we consider anything within +/- 15% a very good score. If I say so myself our job is tougher since we get one shot at a sales number (no monthly revisions) and it can be a year or two out in the future. That said we tell people that while we strive to be 100% accurate what is really important is that we are directionally correct and provide good guidance on go or no go decisions.

  • GiddyHitch GiddyHitch on Jan 04, 2013

    Jessica and her team obviously do great work but I am now wondering how much of her forecasts are driven directly by data gathered from the Edmunds website itself by consumers shopping for a new vehicle. This would be an inside line on potential sales unavailable to other analysts.

Next