When It Comes to Tesla's Accident-reducing Autosteer, Don't Believe the Numbers
There’s a study you should read, and it delivers black eyes to both Tesla and the National Highway Traffic Safety Administration.
You probably remember the fatal crash of a Tesla in Mountain View, California last March, a crash that occurred as the victim’s car cruised along in Autopilot mode. Unexpectedly, the vehicle steered itself out of a lane, impacting a highway divider at high speed. Once again, the effectiveness and safety of Tesla’s Autopilot system came under scrutiny as Tesla scrambled to defend itself. The automaker pointed to the findings of a 2017 NHTSA report released in the wake of a fatal crash from 2016. That study claimed the automaker’s Autosteer system, when introduced as part of the Autopilot suite of automated features, lowered Tesla crash rates by 40 percent.
Don’t believe everything you read, says R.A. Whitfield, director of Quality Control Systems. Whitfield filed a lawsuit and waited nearly two years to get to the bottom of that 40 percent figure.
As the NHTSA didn’t release the dataset behind the study, Whitfield requested it.
“Extraordinary claims ought to be backed by extraordinary evidence,” he told the Los Angeles Times.
Rebuffed, he filed a lawsuit under the Freedom of Information Act against the U.S. Department of Transportation. The data came into his possession in late November of last year.
After looking at the data, Whitfield discovered a serious problem with the methodology behind the controversial 40 percent figure. It has to do with the miles driven before and after Autosteer installation in data provided to the NHTSA by Tesla. The data covers a period from 2014 to 2016.
It’s a long document (you can read it here), but the report issued this month by Quality Control Systems breaks down the problems in the federal road safety agency’s calculations. Information about the number of pre-Autosteer miles traveled by certain vehicles in the data pool is missing, with other vehicles carrying vague info about when exactly Autosteer came online.
The NHTSA’s determination, the Maryland firm claims, was made “by examining the sums of the miles driven prior to Autosteer activation, miles driven after Autosteer activation, airbag deployment events prior to Autosteer activation and airbag deployment events after Autosteer activation for all of the subject vehicles.” That’s a quote from an NHTSA investigator.
In the report’s preamble, the firm states that, based on the incomplete data, it “recognized that NHTSA’s summarization of ‘miles driven prior to Autosteer activation [and] miles driven after Autosteer activation’ might not actually include all of the miles driven before or after Autosteer activation.”
After breaking the data down into different groups of vehicles (based on quality of reported mileage), the firm slammed the NHTSA. “The Agency’s treatment of missing or unreported mileage data in its calculations of exposure mileage as though the mileage were non-existent is not justifiable,” it says in its report.
The vast number of vehicles with missing mileage “results in the inflation of the overall ‘before Autosteer’ airbag deployment crash rate reported by NHTSA, but to a degree that can’t be known with certainty,” the report concludes.
In other words, the 40 percent crash reduction figure doesn’t have a leg to stand on. Exactly by how much Autosteer increased or reduced the crash rate can’t be known, given the incomplete data. The report concludes with a warning for lax agencies and automakers who hope to coax the public into supposedly safe self-driving vehicles.
“A very substantial fraction of the public simply doesn’t trust autonomous driving
technologies. Given the scarcity of scientifically reliable, publicly available
data about the safety of these systems, why should they?”
Latest Car ReviewsRead more
Latest Product ReviewsRead more
- Abqhudson Passenger seating in recent accords has been unacceptable with my 5’2” wife forced to look at the dash while sitting in the hole provided.
- ToolGuy Real Subarus are green and coated with dust from at least three different National Parks (Gateway Arch doesn't count).
- ToolGuy Good for them.(And their customers. $2500 first-year subscription on top of the system cost? That ain't me)
- ToolGuy T E L L U R I D E is not on this list(I can keep my poster on the wall)
- ToolGuy My impression is that Honda has been coasting on its reputation for awhile now.(To Honda's credit, they aren't standing on the Self Destruct button like Toyota seems to be)