Musk's Math: Tackling Tesla's Dubious Autopilot Safety Stats

Steph Willems
by Steph Willems

Tesla CEO Elon Musk vigorously defended his Autopilot system when accident reports rolled in earlier this year. Even when a fatal Florida crash was blamed on a temporarily blinded Autopilot, Musk assured citizens, Tesla owners, regulators and everyone else that the semi-autonomous driving system made his vehicles the safest things on the road.

Just do the math, Musk told the skeptics. Well, someone finally has.

Writing in Green Car Reports, David Noland tackles Musk’s claim that Autopilot-equipped Teslas handily beat America’s conventional vehicle fleet in terms of safety. By unpacking the claim, Noland shows why broad, hazy statistics trotted out as definitive proof often aren’t what they seem.

Critics pounced on Tesla and Musk after the fatal May crash, with many demanding that the system be pulled off the market until it could be proven safe. The National Highway Traffic Safety Administration and National Transportation Safety Bureau launched investigations. Musk shot back with numbers, claiming in a blog post that one fatality in 130 million Autopilot-driven miles beats the U.S. average of one death per 94 million miles driven.

On the outside, that seems pretty clear. Wrong, says Noland. Musk’s sample size — which officially stands at one death per 222 million miles — is just one of the problems with his claim.

“We’re talking here about the smallest possible sample size: one fatality,” Noland writes. And what about that videotaped fatal crash in China earlier this year? If that proves to be Autopilot’s fault, it doubles the system’s fatality rate.

Instantly, the “half a million lucky folks around the world allegedly saved by universal Autopilot would suddenly all be dead again,” Noland writes.

Counting the second fatality and using Musk’s 222 million mile claim, that makes for a fatality rate of one for every 111 million miles driven. That isn’t too far from the 94-million-mile average.

Many proponents of self-driving cars claim automation will reduce deaths, but it’s hard to back up that claim. On the subject of self-driving vehicles, the Rand Corporation claims, “Autonomous vehicles would have to be driven hundreds of millions of miles, and sometimes hundreds of billions of miles, to demonstrate their reliability in terms of of fatalities and injuries.”

That sample size issue again…

Noland then digs deeper into the stats on both sides. Musk apparently gets his 94 million figure from the NHTSA’s Fatality Rate per 100 Million VMT (Vehicle Miles Traveled). And guess what those fatalities also include? Pedestrians, cyclists, bus passengers and truck drivers.

Not only that, passenger car occupants made up only 36 percent of the country’s traffic fatalities last year. A boatload of Teslas could sail through the holes in Musk’s methodology. MIT Technology Review is one of the publications slamming his stats, claiming the comparison “has no meaning.”

There’s other factors keeping Tesla’s fatality rate low. Most Autopilot miles are traveled on major highways, in good weather, with older people behind the wheel (they can afford the vehicle, you see). So, Musk’s stats are drawn from the safest roads, driven in the safest weather, by the safest demographic of drivers.

Measured another way, Tesla’s vehicles are far more dangerous than an average car. The Insurance Institute for Highway Safety uses a different measure: driver deaths per million vehicle-years. The existing American fleet, it says, averages 28 deaths per million vehicle-years.

Using the average number of miles driven by Americans each year, Noland figures that works out to 28 fatalities per 12 billion miles driven, or one death per 428 million miles. Remember Tesla’s rate, assuming the Chinese death is Autopilot’s fault? One fatality per 111 million miles. Four times more dangerous than average.

Maybe Musk should cool it with the safety statistics for now.

[Image: © 2016 David Marek/The Truth About Cars]

Steph Willems
Steph Willems

More by Steph Willems

Comments
Join the conversation
7 of 42 comments
  • MeaMaximaCulpa MeaMaximaCulpa on Oct 13, 2016

    And if we dive deeper in the figures (sh*tty sample size and all) shouldn't the tesla with the autopilot be compared to cars of the same type, age and price? The American car fleet as a whole makes for a horrible comparison does it not? So let's see, the oldest Tesla that's fitted with the hardware required for the autopilot was manufactured exactly two years ago, the Tesla is a luxury sedan (or hatchback) with a base price around, what, $ 75 000.?So is the average car in the group "luxury sedans that were manufactured in the last two years and costing in excess of 75 k when new" safer then the average American car and/or the tesla?

    • Stuki Stuki on Oct 13, 2016

      Of course, you could also take the tack that as long as Autopilot is not obviously less safe than the single least safe human driver that is not banned from the road, why ban Autopilot?

  • RRocket RRocket on Oct 13, 2016

    Or you could use the IIHS methodology which is "per million registered vehicles". When reviewing that data I was extremely shocked to see 9 different vehicles with ZERO deaths for 3 years. Pretty amazing considering some of them are large volume (Lexus RX for example). Anyways, if you use the "per million" calculation, these are all the vehicles that have fewer fatality rate than the Tesla Jeep Compass 2WD Jeep Patriot 4WD Honda Element 4WD Subaru Legacy 4WD Acura TSX Volkswagen CC Audi A4 4WD Acura TL 2WD BMW 328i sedan Mercedes-Benz C-Class Lexus ES 350 Subaru Outback 4WD Volkswagen Jetta Kia Sorento 2WD Toyota Highlander hybrid 4WD Honda Pilot 4WD Chevrolet Equinox 2WD Chevrolet Equinox 4WD Ford Flex 2WD 5 (0-15) Mazda CX-9 4WD Jeep Grand Cherokee 4WD Toyota Highlander 2WD Honda Pilot 2WD Hyundai Santa Fe 4WD Mazda CX-9 2WD Dodge Nitro 4WD Toyota Venza 4WD Lexus RX 350 4WD Volvo XC90 4WD Mercedes-Benz M-Class 4WD Lexus RX 350 2WD Acura MDX 4WD Acura RDX 4WD Lincoln MKX 4WD Dodge Ram 1500 Quad 4WD Chevrolet Silverado 1500 Crew 2WD Honda Ridgeline 4WD Toyota Sequoia 4WD GMC Yukon 4WD Ford Expedition 4WD GMC Yukon 2WD Nissan Armada 2WD Buick Enclave 2WD Chevrolet Traverse GMC Acadia 4WD Buick Enclave 4WD Chevrolet Tahoe 4WD Chevrolet Traverse 4WD Mercedes-Benz GL-Class 4WD Land Rover Range Rover Sport 4WD Honda Odyssey GMC Yukon XL 1500 4WD

    • See 3 previous
    • Orenwolf Orenwolf on Oct 14, 2016

      @jkross22 This is part of why it's critical to get all the data we can, from all the sources we can.

  • MaintenanceCosts It's not a Benz or a Jag / it's a 5-0 with a rag /And I don't wanna brag / but I could never be stag
  • 3-On-The-Tree Son has a 2016 Mustang GT 5.0 and I have a 2009 C6 Corvette LS3 6spd. And on paper they are pretty close.
  • 3-On-The-Tree Same as the Land Cruiser, emissions. I have a 1985 FJ60 Land Cruiser and it’s a beast off-roading.
  • CanadaCraig I would like for this anniversary special to be a bare-bones Plain-Jane model offered in Dynasty Green and Vintage Burgundy.
  • ToolGuy Ford is good at drifting all right... 😉
Next