Uber Whistleblower: Autonomous Vehicles Need New Safety Metrics, Aren't Really Any Safer

Matt Posky
by Matt Posky
uber whistleblower autonomous vehicles need new safety metrics arent really any

Over the past year the automotive industry has carefully walked back the expectations surrounding autonomous cars. Yet pretty much any change in rhetoric constitutes retracted goals. With numerous companies predicting self-driving fleets of commercial vehicles before 2021, the bar couldn’t have been set much higher.

A lack of progress is partly to blame. However, a bundle of high-profile accidents have also shaken public trust — especially after it was found that Uber whistleblower Robbie Miller was trying to alert the company to issues with its self-driving program just days before one of the company’s autonomous Volvos was involved in a fatal accident with a pedestrian.

That’s not the half of it. In April, Miller released a study claiming self-driving vehicles were actually recording incident rates higher than that of your typical motorist. Contrasting data from the Strategic Highway Research Program (SHRP) and the California DMV, he concluded that autonomous test vehicles created more injuries per mile than the average human motorist with a few years of practice.

That’s not what we’re being sold. Automakers have repeatedly suggested that AV testing is a gateway to a safer world, with major breakthroughs close at hand. But Miller argued that focusing on the number of miles a manufacturer covers with its self-driving fleet doesn’t yield much more than reduced public safety.

“I want to make the road safer,” Miller, a former Uber operations manager and current chief safety officer at a California-based company developing advanced driver-assist systems for the trucking industry, explained to Automotive News. “Other programs out there, you know, they’re basically spouting science fiction about things that are years away.”

From Automotive News:

Miller’s caution comes from personal experience. While working as an operations manager at Uber in 2018, he was alarmed that the company’s self-driving test cars were “routinely in accidents resulting in damage” and that collisions occurred “every 15,000 miles.” Those are passages from a prescient email to several Uber executives in which he outlined his concerns and urged a review of the self-driving program, particularly warning that human safety drivers needed to be better trained.

He sent that email on March 13, 2018. Three days later, Miller left his job.

Five days after he sent the email, an Uber self-driving test vehicle struck and killed Elaine Herzberg, a pedestrian in Tempe, Ariz., a death that continues to reverberate through the fledgling industry.

In his study, he alleged that too many automakers and tech firms rely on misleading benchmarks, reframing setbacks, and focus too much on the amount of testing vs technological progress.

“The miles per disengagement metric is a bad metric for measuring progress and is not meaningful in terms of safety. Companies inflate their miles per disengagement to appear further along and use their own absurd definitions of what a disengagement is — effectively erasing thousands of safety-related disengages. Companies that report their disengages honestly appear much further behind. Furthermore, this metric incentivize safety drivers to not disengage in unsafe situations,” he said. “Relying on the wrong metrics or not looking at real data has unfortunately propelled us into the realm of safety theater — meaning creating the illusion of safety instead of actually delivering on safety.”

With Congress likely to move ahead with legislation regarding the safe deployment of self-driving vehicles as early as next month, we’re past due for a critical analysis of the technology. Unfortunately, most of the data given to legislators is coming from tech firms and auto giants with an interest in playing up the positive aspects. A common chorus is that AVs will save anywhere from 3,000 to 37,500 American lives per year. However, it rides on the presumption that they’ll be more effective than human drivers, something that has yet to be proven.

Miller’s fear is that this will taint the metrics used to develop the laws governing autonomous vehicles. For example, automakers are supposed to submit voluntary reports to the National Highway Traffic Safety Administration. One of his core grievances is that “companies misuse this well-intentioned process as a public relations tactic to submit glossy safety brochures with little substance.” Having seen some ourselves, we know he’s not ranting and raving about nothing. Most of these reports read like a sales pamphlet.

He does, however, take time to praise his current employer in his industrial analysis — leaving us wondering how much of it was a sales pitch for Pronto vs a call for sanity. Helmed by Anthony Levandowski, who you’ll remember from the Uber/Waymo intellectual property case, the trucking startup openly touts its commitment to safety and a gradual path toward automated driving.

[Image: NHTSA]

Join the conversation
2 of 18 comments
  • Art Vandelay Art Vandelay on Aug 19, 2019

    You mean the people with the most to gain are overhyping their technology and buying influence with our elected "leaders" to ensure the overhyped tech gets adopted? Boy I am shocked. SHOCKED I say.

  • Snooder Snooder on Aug 20, 2019

    I've said it before and I've said it again. Hyping AV tech as a safety feature was dumb and self defeating from the start. It is NOT safer and never will be. But, it doesnt NEED to be. Nobody buys Cruise Control because its "safe". They buy it because its convenient and makes boring highways miles less shitty. That's what people should be touting. Cars that park themselves. A slightly less dull commute. Automatic braking. Turn signals that kick on when your off ramp is coming up. Maybe the ability to have your car pull up from the parking lot so you dont have to run out into the rain. Etc, etc. Just incremental improvements in the driving experience.

  • Nrd515 I bought an '88 S10 Blazer with the 4.3. We had it 4 years and put just about 48K on it with a bunch of trips to Nebraska and S. Dakota to see relatives. It had a couple of minor issues when new, a piece of trim fell off the first day, and it had a seriously big oil leak soon after we got it. The amazinly tiny starter failed at about 40K, it was fixed under some sort of secret warranty and we got a new Silverado as a loaner. Other than that, and a couple of tires that blew when I ran over some junk on the road, it was a rock. I hated the dash instrumentation, and being built like a gorilla, it was about an inch and a half too narrow for my giant shoulders, but it drove fine, and was my second most trouble free vehicle ever, only beaten by my '82 K5 Blazer, which had zero issues for nearly 50K miles. We sold the S10 to a friend, who had it over 20 years and over 400,000 miles on the original short block! It had a couple of transmissions, a couple of valve jobs, a rear end rebuild at 300K, was stolen and vandalized twice, cut open like a tin can when a diabetic truck driver passed out(We were all impressed at the lack of rust inside the rear quarters at almost 10 years old, and it just went on and on. Ziebart did a good job on that Blazer. All three of his sons learned to drive in it, and it was only sent to the boneyard when the area above the windshield had rusted to the point it was like taking a shower when it rained. He now has a Jeep that he's put a ton of money into. He says he misses the S10's reliablity a lot these days, the Jeep is in the shop a lot.
  • Jeff S Most densely populated areas have emission testing and removing catalytic converters and altering pollution devices will cause your vehicle to fail emission testing which could effect renewing license plates. In less populated areas where emission testing is not done there would probably not be any legal consequences and the converter could either be removed or gutted both without having to buy specific parts for bypassing emissions. Tampering with emission systems would make it harder to resell a vehicle but if you plan on keeping the vehicle and literally running it till the wheels fall off there is not much that can be done if there is no emission testing. I did have a cat removed on a car long before mandatory emission testing and it did get better mpgs and it ran better. Also had a cat gutted on my S-10 which was close to 20 years old which increased performance and efficiency but that was in a state that did not require emission testing just that reformulated gas be sold during the Summer months. I would probably not do it again because after market converters are not that expensive on older S-10s compared to many of the newer vehicles. On newer vehicles it can effect other systems that are related to the operating and the running of the vehicle. A little harder to defeat pollution devices on newer vehicles with all the systems run by microprocessors but if someone wants to do it they can. This law could be addressing the modified diesels that are made into coal rollers just as much as the gasoline powered vehicles with cats. You probably will still be able to buy equipment that would modify the performance of a vehicles as long as the emission equipment is not altered.
  • ToolGuy I wonder if Vin Diesel requires DEF.(Does he have issues with Sulfur in concentrations above 15ppm?)
  • ToolGuy Presented for discussion: https://xroads.virginia.edu/~Hyper2/thoreau/civil.html
  • Kevin Ford can do what it's always done. Offer buyouts to retirement age employees, and transfers to operating facilities to those who aren't retirement age. Plus, the transition to electric isn't going to be a finger snap one time event. It's going to occur over a few model years. What's a more interesting question is: Where will today's youth find jobs in the auto industry given the lower employment levels?