Uber Whistleblower: Autonomous Vehicles Need New Safety Metrics, Aren't Really Any Safer

Over the past year the automotive industry has carefully walked back the expectations surrounding autonomous cars. Yet pretty much any change in rhetoric constitutes retracted goals. With numerous companies predicting self-driving fleets of commercial vehicles before 2021, the bar couldn’t have been set much higher.

A lack of progress is partly to blame. However, a bundle of high-profile accidents have also shaken public trust — especially after it was found that Uber whistleblower Robbie Miller was trying to alert the company to issues with its self-driving program just days before one of the company’s autonomous Volvos was involved in a fatal accident with a pedestrian.

That’s not the half of it. In April, Miller released a study claiming self-driving vehicles were actually recording incident rates higher than that of your typical motorist. Contrasting data from the Strategic Highway Research Program (SHRP) and the California DMV, he concluded that autonomous test vehicles created more injuries per mile than the average human motorist with a few years of practice.

That’s not what we’re being sold. Automakers have repeatedly suggested that AV testing is a gateway to a safer world, with major breakthroughs close at hand. But Miller argued that focusing on the number of miles a manufacturer covers with its self-driving fleet doesn’t yield much more than reduced public safety.

“I want to make the road safer,” Miller, a former Uber operations manager and current chief safety officer at a California-based company developing advanced driver-assist systems for the trucking industry, explained to Automotive News. “Other programs out there, you know, they’re basically spouting science fiction about things that are years away.”

From Automotive News:

Miller’s caution comes from personal experience. While working as an operations manager at Uber in 2018, he was alarmed that the company’s self-driving test cars were “routinely in accidents resulting in damage” and that collisions occurred “every 15,000 miles.” Those are passages from a prescient email to several Uber executives in which he outlined his concerns and urged a review of the self-driving program, particularly warning that human safety drivers needed to be better trained.

He sent that email on March 13, 2018. Three days later, Miller left his job.

Five days after he sent the email, an Uber self-driving test vehicle struck and killed Elaine Herzberg, a pedestrian in Tempe, Ariz., a death that continues to reverberate through the fledgling industry.

In his study, he alleged that too many automakers and tech firms rely on misleading benchmarks, reframing setbacks, and focus too much on the amount of testing vs technological progress.

“The miles per disengagement metric is a bad metric for measuring progress and is not meaningful in terms of safety. Companies inflate their miles per disengagement to appear further along and use their own absurd definitions of what a disengagement is — effectively erasing thousands of safety-related disengages. Companies that report their disengages honestly appear much further behind. Furthermore, this metric incentivize safety drivers to not disengage in unsafe situations,” he said. “Relying on the wrong metrics or not looking at real data has unfortunately propelled us into the realm of safety theater — meaning creating the illusion of safety instead of actually delivering on safety.”

With Congress likely to move ahead with legislation regarding the safe deployment of self-driving vehicles as early as next month, we’re past due for a critical analysis of the technology. Unfortunately, most of the data given to legislators is coming from tech firms and auto giants with an interest in playing up the positive aspects. A common chorus is that AVs will save anywhere from 3,000 to 37,500 American lives per year. However, it rides on the presumption that they’ll be more effective than human drivers, something that has yet to be proven.

Miller’s fear is that this will taint the metrics used to develop the laws governing autonomous vehicles. For example, automakers are supposed to submit voluntary reports to the National Highway Traffic Safety Administration. One of his core grievances is that “companies misuse this well-intentioned process as a public relations tactic to submit glossy safety brochures with little substance.” Having seen some ourselves, we know he’s not ranting and raving about nothing. Most of these reports read like a sales pamphlet.

He does, however, take time to praise his current employer in his industrial analysis — leaving us wondering how much of it was a sales pitch for Pronto vs a call for sanity. Helmed by Anthony Levandowski, who you’ll remember from the Uber/Waymo intellectual property case, the trucking startup openly touts its commitment to safety and a gradual path toward automated driving.

[Image: NHTSA]