When It Comes to Tesla's Accident-reducing Autosteer, Don't Believe the Numbers

when it comes to teslas accident reducing autosteer dont believe the numbers

There’s a study you should read, and it delivers black eyes to both Tesla and the National Highway Traffic Safety Administration.

You probably remember the fatal crash of a Tesla in Mountain View, California last March, a crash that occurred as the victim’s car cruised along in Autopilot mode. Unexpectedly, the vehicle steered itself out of a lane, impacting a highway divider at high speed. Once again, the effectiveness and safety of Tesla’s Autopilot system came under scrutiny as Tesla scrambled to defend itself. The automaker pointed to the findings of a 2017 NHTSA report released in the wake of a fatal crash from 2016. That study claimed the automaker’s Autosteer system, when introduced as part of the Autopilot suite of automated features, lowered Tesla crash rates by 40 percent.

Don’t believe everything you read, says R.A. Whitfield, director of Quality Control Systems. Whitfield filed a lawsuit and waited nearly two years to get to the bottom of that 40 percent figure.

As the NHTSA didn’t release the dataset behind the study, Whitfield requested it.

“Extraordinary claims ought to be backed by extraordinary evidence,” he told the Los Angeles Times.

Rebuffed, he filed a lawsuit under the Freedom of Information Act against the U.S. Department of Transportation. The data came into his possession in late November of last year.

After looking at the data, Whitfield discovered a serious problem with the methodology behind the controversial 40 percent figure. It has to do with the miles driven before and after Autosteer installation in data provided to the NHTSA by Tesla. The data covers a period from 2014 to 2016.

It’s a long document (you can read it here), but the report issued this month by Quality Control Systems breaks down the problems in the federal road safety agency’s calculations. Information about the number of pre-Autosteer miles traveled by certain vehicles in the data pool is missing, with other vehicles carrying vague info about when exactly Autosteer came online.

The NHTSA’s determination, the Maryland firm claims, was made “by examining the sums of the miles driven prior to Autosteer activation, miles driven after Autosteer activation, airbag deployment events prior to Autosteer activation and airbag deployment events after Autosteer activation for all of the subject vehicles.” That’s a quote from an NHTSA investigator.

In the report’s preamble, the firm states that, based on the incomplete data, it “recognized that NHTSA’s summarization of ‘miles driven prior to Autosteer activation [and] miles driven after Autosteer activation’ might not actually include all of the miles driven before or after Autosteer activation.”

After breaking the data down into different groups of vehicles (based on quality of reported mileage), the firm slammed the NHTSA. “The Agency’s treatment of missing or unreported mileage data in its calculations of exposure mileage as though the mileage were non-existent is not justifiable,” it says in its report.

The vast number of vehicles with missing mileage “results in the inflation of the overall ‘before Autosteer’ airbag deployment crash rate reported by NHTSA, but to a degree that can’t be known with certainty,” the report concludes.

In other words, the 40 percent crash reduction figure doesn’t have a leg to stand on. Exactly by how much Autosteer increased or reduced the crash rate can’t be known, given the incomplete data. The report concludes with a warning for lax agencies and automakers who hope to coax the public into supposedly safe self-driving vehicles.

“A very substantial fraction of the public simply doesn’t trust autonomous driving


technologies. Given the scarcity of scientifically reliable, publicly available


data about the safety of these systems, why should they?”

[Image: Tesla]

Comments
Join the conversation
3 of 19 comments
  • ToddAtlasF1 ToddAtlasF1 on Feb 15, 2019

    Why do people want more government? The people involved can never be trusted with the power leftists want them to have. AVs suit their agendas, so we're going to be told they're ready whether or not they are and then the world will be diminished to the level of their actual capabilities.

    • Vulpine Vulpine on Feb 15, 2019

      Off topic. It doesn't matter why, what matters is the result. Think about it.

  • HotPotato HotPotato on Feb 20, 2019

    As the commenter above aptly put it, lane keep assist is an aid for tired or distracted drivers, not an invitation to take your hands off the wheel. Tesla still gets three raspberries from me for sticking with the "Autopilot" brand name, but in fairness they've made it much clearer to drivers of late that they must keep their hands on the wheel and take primary responsibility for driving, or the car will scream at them and turn the feature off.

  • FreedMike I just don’t see the market here - I think about 1.2% of Jeep drivers are going to be sold on the fuel cost savings here. And the fuel cost savings are pretty minimal, per the EPA: https://www.fueleconomy.gov/feg/PowerSearch.do?action=noform&path=1&year1=2022&year2=2022&make=Jeep&baseModel=Wrangler&srchtyp=ymm&pageno=1&rowLimit=50Annual fuel costs for this vehicle are $2200 and $2750 for the equivalent base turbo-four model. I don’t get it.
  • FreedMike How about the “Aztek” package? Wait, this car already has that…Said it before and I’ll say it again: they need to restyle the hind end on this car, stat.
  • Johnster "Vale" is the [s]cheap[/s] lower-priced performance version with black trim and stiff suspension."Mist" is the "DeLuxe" version with a bit more chrome and trim. (Sort of like the "Decor Package" option.)"Magentic" is the full-on Brougham treatment (in its current state) with more chrome trim than the "Mist" and all sorts of gimmicky electronic features inside. (Sadly, it will not include simulated landau irons or a vinyl covered roof, even as an option.)"Aurora" is the Oldsmobile of Cadillacs (sort of like the old Cadillac Calais). No, that's not right. It's the top-of-the-line model, sort of a "Grand Touring" version, with not as much chrome as the "Magentic" but all of the gimmicky electronic features and a stiffer suspension.
  • Drew8MR Why can't CARB leave hobbyists alone? Maybe lay off the low hanging fruit and go after the gross polluters. Bring back the rolling exemption.
  • ToolGuy According to Americans, the very lovely and quite powerful Ford Fairmont (1978-1983) was Way Better.Source: Sales figures.
Next