How Safe Is Level 2 Autonomy? As Safe as the Automaker Makes It

No longer content with just crashing vehicles into walls or poles, the Insurance Institute for Highway Safety has turned its attention to the ever-expanding buffet of new technologies found in today’s automobiles. Driver assistance features ease the driver’s workload and make for comfortable highway cruising, but all systems aren’t made equal.

For its test of “Level 2” autonomous features — meaning, specifically, adaptive cruise control (ACC) and active lane-holding — IIHS selected five premium four-doors. The test group consisted of a 2017 BMW 5 Series with Driving Assistant Plus, a 2017 Mercedes-Benz E-Class with Drive Pilot, a 2018 Tesla Model 3 and 2016 Model S equipped with Autopilot, and a 2018 Volvo S90 with Pilot Assist. Given that each model garnered a “superior” IIHS score for automatic emergency braking, they seemed like choice candidates for a comparison test.

The results, as you’d probably expect, were a mixed bag. We’re not at the technological finish line yet; not by a long shot.

“The new tests are an outgrowth of our research on Level 2 autonomy,” said Jessica Jermakian, IIHS senior research engineer, in a statement. “We zeroed in on situations our staff have identified as areas of concern during test drives with Level 2 systems, then used that feedback to develop road and track scenarios to compare vehicles.”

IIHS wanted to see how the different adaptive cruise control systems handled a stopped lead vehicle, and what would happen when a lead vehicle exited the lane.

First, the vehicles had their ACC turned off, then driven at 31 mph towards a stopped “vehicle” (target). The automatic emergency braking in all but the Teslas stopped the vehicle before reaching the target. The non-profit organization then turned ACC on and repeated the tests with the system’s following distance set to close, middle, and far.

Only the S90 braked abruptly with ACC turned on, while the others braked gradually; each avoiding the stopped vehicle. IIHS noticed the two Teslas began braking earlier than their foreign competitors.

Next up was a test where a lead vehicle braked to a stop, then resumed its journey. Top marks for all vehicles, which braked gradually to match the lead vehicle’s speed.

The complexity of the tests then ramped up. Recent crashes involving Tesla vehicles driving with Autopilot engaged — and a distracted driver behind the wheel — suggest the system can be tricked by a lead vehicle changing lanes, revealing a stopped vehicle ahead of it. IIHS performed this test on all five cars. The distance to the inflatable target vehicle revealed by the lane-changing lead vehicle allowed for 4.3 seconds of travel time before collision.

Much to your surprise, perhaps, all five vehicles applied the brakes and stopped before reaching the target vehicle. The S90 repeated its panic stop, but never did the five cars touch the target.

As the testing conditions were near-perfect in terms of road surface and lighting, IIHS concluded that, “Under ideal conditions, advanced driver assistance systems may function better than they do in more complex driving situations.”

Browsing through the owner’s manuals revealed warnings that vehicles operating on ACC might not recognize and brake for a stationary vehicle in the driving lane. Some IIHS testers recall four of the five models — the Model 3 being the exception — behaving in a similar manner while out and about. While driving the Model 3 on regular roads, researchers recorded 12 instances (over 180 miles) when the car braked gradually to avoid non-existent obstacles. These were either shadows crossing the road, or oncoming vehicles.

“At IIHS we are coached to intervene without warning, but other drivers might not be as vigilant,” Jermakian said. “ACC systems require drivers to pay attention to what the vehicle is doing at all times and be ready to brake manually.”

ACC systems that behave erratically could lead to crashes in heavy traffic. As well, they might persuade drivers to avoid using it, thus potentially putting the driver and vehicle in unnecessary danger. That’s assuming the vehicles are safer with the systems turned on than not.

Regardless, the IIHS tests show that improvements are required to bring Level 2 autonomy up to snuff.

Tests of the vehicles’ active lane-keeping systems also revealed some quirks. On a test course with hills, curves, and well-marked lanes, the systems — each capable of following a lead vehicle when lane markers are obscured — were supposed to keep the vehicle centered in its lane, no matter what. With the exception of the Model 3, this didn’t happen.

IIHS tested each vehicle via six different trials on three stretches of road:

Only the Model 3 stayed within the lane on all 18 trials. The Model S was similar but overcorrected on one curve, causing it to cross the line on the inside of the curve in one trial. None of the other systems tested provided enough steering input on their own to consistently stay in their lane, often requiring the driver to provide additional steering to successfully navigate the curve.

The E-Class stayed within the lane in 9 of 17 runs and strayed to the lane marker in five trials. The system disengaged itself in one trial and crossed the line in two. The 5 series stayed within the lane in 3 of 16 trials and was more likely to disengage than steer outside the lane. The S90 stayed in the lane in 9 of 17 runs and crossed the lane line in eight runs.

Noticing that such systems are often “flummoxed by hills,” the testers threw hills at all five vehicles. Only the Bimmer and Model 3 impressed:

The E-Class stayed in its lane in 15 of 18 trials and on the line in one trial, continuously providing steering support without erratic moves when lane lines weren’t visible. The Model 3 also stayed in the lane in all but one trial, when it hugged the line.

In contrast, the 5-series, Model S and S90 struggled. The 5-series steered toward or across the lane line regularly, requiring drivers to override the steering support to get it back on track. Sometimes the car disengaged steering assistance on its own. The car failed to stay in the lane on all 14 valid trials.

The Model S was errant in the hill tests, staying in the lane in 5 of 18 trials. When cresting hills, the Model S swerved left and right until it determined the correct place in the lane, jolting test drivers. It rarely warned them to take over as it hunted for the lane center. The car regularly veered into the adjacent lanes or onto the shoulder.

When drivers intervened to avoid potential trouble, the active lane-keeping system disengaged. Steering assistance only resumed after drivers re-engaged Autopilot.

The S90 stayed in the lane in 9 of 16 trials. The car crossed the lane line in two trials and in four trials disengaged steering assistance when it crested hills but automatically re-engaged when the system once again detected the markings.

More so than adaptive cruise control or automatic emergency braking, lane-holding features seem to offer the greatest likelihood of error. Keeping one’s hands on the wheel and eyes on the road is the only way to mitigate the danger.

It’s too bad Cadillac’s Super Cruise wasn’t also on trial here. Regardless, the IIHS testing shows that, while driver assist features have come a long way in recent years, we shouldn’t expect full autonomy in showrooms in the near future. It also makes crystal clear why we should never refer to Level 2 autonomy as “self-driving.”

[Image: Daimler AG]