By on August 7, 2018

No longer content with just crashing vehicles into walls or poles, the Insurance Institute for Highway Safety has turned its attention to the ever-expanding buffet of new technologies found in today’s automobiles. Driver assistance features ease the driver’s workload and make for comfortable highway cruising, but all systems aren’t made equal.

For its test of “Level 2” autonomous features — meaning, specifically, adaptive cruise control (ACC) and active lane-holding — IIHS selected five premium four-doors. The test group consisted of a 2017 BMW 5 Series with Driving Assistant Plus, a 2017 Mercedes-Benz E-Class with Drive Pilot, a 2018 Tesla Model 3 and 2016 Model S equipped with Autopilot, and a 2018 Volvo S90 with Pilot Assist. Given that each model garnered a “superior” IIHS score for automatic emergency braking, they seemed like choice candidates for a comparison test.

The results, as you’d probably expect, were a mixed bag. We’re not at the technological finish line yet; not by a long shot.

“The new tests are an outgrowth of our research on Level 2 autonomy,” said Jessica Jermakian, IIHS senior research engineer, in a statement. “We zeroed in on situations our staff have identified as areas of concern during test drives with Level 2 systems, then used that feedback to develop road and track scenarios to compare vehicles.”

IIHS wanted to see how the different adaptive cruise control systems handled a stopped lead vehicle, and what would happen when a lead vehicle exited the lane.

First, the vehicles had their ACC turned off, then driven at 31 mph towards a stopped “vehicle” (target). The automatic emergency braking in all but the Teslas stopped the vehicle before reaching the target. The non-profit organization then turned ACC on and repeated the tests with the system’s following distance set to close, middle, and far.

Only the S90 braked abruptly with ACC turned on, while the others braked gradually; each avoiding the stopped vehicle. IIHS noticed the two Teslas began braking earlier than their foreign competitors.

Next up was a test where a lead vehicle braked to a stop, then resumed its journey. Top marks for all vehicles, which braked gradually to match the lead vehicle’s speed.

The complexity of the tests then ramped up. Recent crashes involving Tesla vehicles driving with Autopilot engaged — and a distracted driver behind the wheel — suggest the system can be tricked by a lead vehicle changing lanes, revealing a stopped vehicle ahead of it. IIHS performed this test on all five cars. The distance to the inflatable target vehicle revealed by the lane-changing lead vehicle allowed for 4.3 seconds of travel time before collision.

Much to your surprise, perhaps, all five vehicles applied the brakes and stopped before reaching the target vehicle. The S90 repeated its panic stop, but never did the five cars touch the target.

As the testing conditions were near-perfect in terms of road surface and lighting, IIHS concluded that, “Under ideal conditions, advanced driver assistance systems may function better than they do in more complex driving situations.”

Browsing through the owner’s manuals revealed warnings that vehicles operating on ACC might not recognize and brake for a stationary vehicle in the driving lane. Some IIHS testers recall four of the five models — the Model 3 being the exception — behaving in a similar manner while out and about. While driving the Model 3 on regular roads, researchers recorded 12 instances (over 180 miles) when the car braked gradually to avoid non-existent obstacles. These were either shadows crossing the road, or oncoming vehicles.

“At IIHS we are coached to intervene without warning, but other drivers might not be as vigilant,” Jermakian said. “ACC systems require drivers to pay attention to what the vehicle is doing at all times and be ready to brake manually.”

ACC systems that behave erratically could lead to crashes in heavy traffic. As well, they might persuade drivers to avoid using it, thus potentially putting the driver and vehicle in unnecessary danger. That’s assuming the vehicles are safer with the systems turned on than not.

Regardless, the IIHS tests show that improvements are required to bring Level 2 autonomy up to snuff.

Tests of the vehicles’ active lane-keeping systems also revealed some quirks. On a test course with hills, curves, and well-marked lanes, the systems — each capable of following a lead vehicle when lane markers are obscured — were supposed to keep the vehicle centered in its lane, no matter what. With the exception of the Model 3, this didn’t happen.

IIHS tested each vehicle via six different trials on three stretches of road:

Only the Model 3 stayed within the lane on all 18 trials. The Model S was similar but overcorrected on one curve, causing it to cross the line on the inside of the curve in one trial. None of the other systems tested provided enough steering input on their own to consistently stay in their lane, often requiring the driver to provide additional steering to successfully navigate the curve.

The E-Class stayed within the lane in 9 of 17 runs and strayed to the lane marker in five trials. The system disengaged itself in one trial and crossed the line in two. The 5 series stayed within the lane in 3 of 16 trials and was more likely to disengage than steer outside the lane. The S90 stayed in the lane in 9 of 17 runs and crossed the lane line in eight runs.

Noticing that such systems are often “flummoxed by hills,” the testers threw hills at all five vehicles. Only the Bimmer and Model 3 impressed:

The E-Class stayed in its lane in 15 of 18 trials and on the line in one trial, continuously providing steering support without erratic moves when lane lines weren’t visible. The Model 3 also stayed in the lane in all but one trial, when it hugged the line.

In contrast, the 5-series, Model S and S90 struggled. The 5-series steered toward or across the lane line regularly, requiring drivers to override the steering support to get it back on track. Sometimes the car disengaged steering assistance on its own. The car failed to stay in the lane on all 14 valid trials.

The Model S was errant in the hill tests, staying in the lane in 5 of 18 trials. When cresting hills, the Model S swerved left and right until it determined the correct place in the lane, jolting test drivers. It rarely warned them to take over as it hunted for the lane center. The car regularly veered into the adjacent lanes or onto the shoulder.

When drivers intervened to avoid potential trouble, the active lane-keeping system disengaged. Steering assistance only resumed after drivers re-engaged Autopilot.

The S90 stayed in the lane in 9 of 16 trials. The car crossed the lane line in two trials and in four trials disengaged steering assistance when it crested hills but automatically re-engaged when the system once again detected the markings.

More so than adaptive cruise control or automatic emergency braking, lane-holding features seem to offer the greatest likelihood of error. Keeping one’s hands on the wheel and eyes on the road is the only way to mitigate the danger.

It’s too bad Cadillac’s Super Cruise wasn’t also on trial here. Regardless, the IIHS testing shows that, while driver assist features have come a long way in recent years, we shouldn’t expect full autonomy in showrooms in the near future. It also makes crystal clear why we should never refer to Level 2 autonomy as “self-driving.”

[Image: Daimler AG]

Get the latest TTAC e-Newsletter!

Recommended

31 Comments on “How Safe Is Level 2 Autonomy? As Safe as the Automaker Makes It...”


  • avatar
    dwford

    It’s a little shocking that these cars failed at so many points on these “ideal circumstances” tests. If the automakers let these safety systems out the door with this level of errors, what is really the point of having them in the car?

    I don’t think this test is an appropriate test for Super Cruise. Isn’t that designed to work only on limited access highways (that Cadillac has mapped)?

    • 0 avatar
      Lockstops

      You are right: it is pointless to have them in the car. Just a gimmick to try out a few times. Otherwise I’m 100% sure they are more of a danger. But they really are just a bonus, it costs basically nothing to throw in the software for them. The auto steering is there anyway too in modern cars: they come along as a bonus with electric power steering. What does cost is the cameras and other sensors, but those are handy for the automatic emergency braking system at low speeds, the only good use for all this stuff. And you get a worse version of adaptive cruise too with the cameras. Top tip: if you get adaptive cruise, make sure it’s radar-based and definitely NOT camera based tech.

      I have Driving Assistant Plus in my BMW, and I have no use for it. It’s especially aggravating that I completely lose any kind of cruise control whenever the sun is very low, blinding the cameras or there is spray coming off the wet roads: then the system makes the cruise control turn off and unable to re-engage (no way to use a traditional non-adaptive cruise either).

      • 0 avatar
        AKM

        Once the software is developed, it costs nothing to throw it in. But developing it is stupendously costly, for (so far) pretty poor results. And the more sensors, the better, but those are expensive too.

      • 0 avatar
        sgeffe

        Interesting..I thought most ACCs had an option to revert to conventional cruise control if desired.

        I agree with your assertion about radar-based ACC. My 2013 Accord Touring uses Radar, and it’s pretty good, though the later Honda systems are even better (smoother, less abrupt braking and quicker recovery to speed if traffic clears ahead), and they might use cameras to augment the radar.

    • 0 avatar
      MBella

      It looks like most of these systems worked as designed. They didn’t cross the line. (With the exception of the Volvo strangely.) With the exception of Tesla, they are not made to drive the vehicle for you, just keep the car bouncing between the lines. The systems disengage when they perceive that the driver is relying on it to much.

  • avatar
    Chocolatedeath

    Is what he did even legal?

    • 0 avatar
      Garrett

      If it doesn’t happen, the shareholder lawsuits definitely will

      • 0 avatar
        civicjohn

        @Garrett, traders are very unhappy, there is a blog out there that is giving investors the blow-by-blow of EM’s tweets, you can get an idea here:

        https://www.msn.com/en-us/money/companies/tesla-says-no-final-decision-has-been-made-to-take-company-private/ar-BBLCX4c

        The trading halt has stopped, the email he just posted seemed to be a walk-back, but the stock is up $10 since the trading halt has been stopped. It kind of looks like this is his “short squeeze”, but I have no idea if what he just did is legal. I guess we’ll see, as it hasn’t got close to the number that he proposed in taking the company private at $420 per share and is trading lower in after hours.

        Here is the actual blog:
        https://www.tesla.com/blog/taking-tesla-private?mod=article_inline

  • avatar
    TrailerTrash

    it had better perform since its the only car that requires you to glance away from the road to perform simple duties.

  • avatar
    TrailerTrash

    although i LOVE the safety systems being put into cars these days, especially since I believe regulations are forcing less and less view, there is never going to be full auto.
    At least not until all roads and cars are able to recognize all conditions and cars talk to each other.

    and in many cases, i do NOT want a car over-riding or slam braking on my decision to abruptly leave my lane to avoid smashing into another car or object.
    IF I decide there is enough room and it MUST be done to save MY life…bugger off AI system.

  • avatar
    seth1065

    Little help here did the Telsa hit the stopped target? The sentence does not seem to say? My fear is folks take self driving as gospel and depend on it to much.

    First, the vehicles had their ACC turned off, then driven at 31 mph towards a stopped “vehicle” (target). The automatic emergency braking in all but the Teslas stopped the vehicle before reaching the target.

    • 0 avatar
      Tele Vision

      Both Teslas hit it. Here ya go!

      ‘Among the scariest found by the Virginia-based institute was with the system in two Tesla vehicles, the Model S and Model 3. The institute tested the system with the adaptive cruise control turned off, but automatic braking on. At 31 miles per hour, both Teslas braked and mitigated a crash but still hit a stationary balloon. They were the only two models that failed to stop in time during tests on a track.’

      The Insurance Institute for Highway Safety, in a paper titled “Reality Check,” issued the warning Tuesday after testing five of the systems from Tesla, Mercedes, BMW and Volvo on a track and public roads. The upshot is while they could save your life, the systems can fail under many circumstances.

      “We have found situations where the vehicles under semi-automated control may do things that can put you and your passengers at risk, and so you really need to be on top of it to prevent that from happening,” said David Zuby, the institute’s chief research officer.

      https://www.ctvnews.ca/autos/testing-finds-flaws-with-car-electronic-car-safety-systems-1.4042916

    • 0 avatar
      aajax

      It’s strange that the Tesla was the first to brake in the ACC test, yet failed to stop in time for the FCA test. Different sensor systems? Or is Tesla concerned that braking too forcefully risks being hit from the rear and losing control?

  • avatar
    seth1065

    If Elon can go private it would be in everyones interest , they need to get out of the Wall Street and public glare while they learn how to make cars in a larger amount. I do not think they will survive buring cash as they are, as a stand alone company , but maybe they can pull it off.

  • avatar
    civicjohn

    @big 3, never thought I’d see a post from you here, you’re really expanding your boundaries from Electrek, what with your turning the conversation into politics “this week”.

    Yup, read the R&T review, good thing they brought 2 cars. Saw the jump in the stock (some people actually have 2 screens attached to their computer, and amazingly enough, there are applications out there that even alert the user of a stock that is moving – I swear, it actually exists)!

    Interesting times indeed. I surely hope he has landed the $$ to take the company private, because the SEC frowns upon some of the things EM has done recently, and no one on the last earnings call had the balls to ask him if there is an ongoing SEC investigation. As I’m sure you are a dual-screen dude like myself, you already know that trading has been halted. Could an oil-rich Saudi investor pulled the trigger to take the company private? “Funding secured”. If he has that in place, then all of the daily click-bait will go away, no more Bloomberg Tracker, no VIN counters, nothing.

    Reminds me of when Michael Dell took his company private. If Tesla goes private, then the valuation truly is based on execution. Ship $35k Model 3s, build Gigafactories, produce the Semi, Roadster, Model Y, flamethrowers, whatever.

    I’d like to see it happen, if for no other reason than removing his tweets from the daily news stream.

  • avatar
    SCE to AUX

    “It also makes crystal clear why we should never refer to Level 2 autonomy as “self-driving.””

    Exactly. As I’ve been saying all along, as long as Level 2 systems require driver vigilance, they don’t even have to work.

  • avatar
    stingray65

    If the testing was done at 31 mph, then it does not simulate all the Tesla accidents where the car has been travelling at 60-75 mph on auto-pilot with the “driver” not paying attention in the time leading up to impact. Obviously stopping from 60 takes a lot more distance than stopping from 30.

    I’ve been helping a friend car shop recently, and it is interesting how few of the premium brands stocked at the dealership have ACC and the full array of “auto-pilot” stuff installed. It seems that the dealers do not believe most customers want to pay the price for these extras.

    • 0 avatar
      aajax

      A lot of people do want ACC, but that feature is at the top of the pyramid. You usually have to buy $5000 worth of stuff you may not want in order to get it. (Good for Toyota for making a complete package available as standard on Corollas, or a reasonable option on others.)

  • avatar
    srh

    I’ve had a few cars with some of these systems. The only one I’ve found useful thus far is the ACC. When staying /in one lane/ driving at a leisurely pace, I appreciate that it performs the usual task of trying to match the vehicle’s speed ahead of me.

    That said, under any other circumstances it can cause trouble. I remember once when driving in the slow lane and catching up to a semi, I put on my blinker to change lanes. Plenty of space between me and the semi. Right as I’m changing lanes the ACC slows me down considerably to match the semi’s speed. I’m sure the cars in the fast lane weren’t thrilled by the bozo changing to their lane and smashing his brake pedal at the same time.

    So yeah, I pretty much only use it when I’m driving at the speed limit, and these days I disengage cruise control before changing lanes. Not sure that gains me a whole lot over just having normal cruise control…

    • 0 avatar
      sgeffe

      I’ve found that you need to be watching the rear view while on ACC, and be ready to give a touch of throttle to override a possible “brake grenade.” It only took me a week or so to figure out the nuances, and it gets easier with practice.

      The ability to let the car pace traffic, instead of disengaging and reengaging cruise, is worth the occasional software stumble. Your next car’s system will probably be leaps and bounds over the one in your present car.

    • 0 avatar
      aajax

      Most of the minor roads here in New Jersey are solid no-passing zones, with drivers who can’t maintain speed. So ACC is a sanity-saver.

  • avatar
    Lockstops

    The only ways to ever have ‘win’ and ‘Tesla’ in the same sentence is by being able to _sell_ overpriced Tesla stock or by shorting it.

  • avatar
    SPPPP

    I am starting to get nostalgic for the days when BTSR had to post first in every article … and that worries me.

  • avatar
    Tele Vision

    ‘Driver assistance features ease the driver’s workload and make for comfortable highway cruising.’

    You mean ‘Lull into a false sense of security while travelling at 100 FPS in a rolling incendiary device comprised of parts and technology supplied by the lowest bidder.’

    What happened to just, I dunno, driving? How did driving get so difficult? I can’t wait to see what all these ‘safety’ ‘improvements’ do to my insurance premiums – likely just what ABS/DRLs/TCS/SCS did. Bugger all.

    I’ve recently been resigned to buying old cars and trucks. Now they’re just going to be older. And older.

  • avatar
    AKM

    @Steph Williams:

    “Noticing that such systems are often “flummoxed by hills,” the testers threw hills at all five vehicles. Only the Bimmer and Model 3 impressed:

    The E-Class stayed in its lane in 15 of 18 trials and on the line in one trial, continuously providing steering support without erratic moves when lane lines weren’t visible. The Model 3 also stayed in the lane in all but one trial, when it hugged the line.”

    I think you meant the Benz, not the Bimmer, in your first sentence.

    I haven’t done a spreadsheet comparison of the various results, but I got the feeling that overall, the Volvo was a little behind, and the Tesla 3 the best of the bunch by a small margin.
    Anyone ran the numbers more precisely?

  • avatar
    SSJeep

    This comparison test would have been MUCH better if more manufacturers were represented. The omission of SuperCruise is unacceptable, as SC is supposedly the industry leading product. But what about Subaru EyeSight, Jeep ASC/Lane Keeping/FCB+, etc? Id even pay to see a write-up feathering all manufacturers, as these features are key to my car buying decisions.

  • avatar
    rtr

    Gadgets designed to allow people addicted to smartphones to network while driving. What could possibly go wrong ….

  • avatar
    aajax

    People should at least learn to use text-to-speech and voice to text features, so they don’t have to look at the damn gadget. Loved this by the way (see to end): https://www.youtube.com/watch?v=O51f1BZKPoo

Read all comments

Back to TopLeave a Reply

You must be logged in to post a comment.

Recent Comments

  • ToolGuy: Ford did model changeover in 2019 (were there any hiccups? I don’t remember). Anyway, Ford has nothing...
  • highdesertcat: Yo! I had a Grand Wagoneer and no white walls would have fixed everything that was wrong with that...
  • highdesertcat: Vulpine, different strokes for different folks. That’s what makes the world go ’round....
  • highdesertcat: Lie2me, there are some sweet deals to be had on 2020 Grand Cherokees right now. A couple of young AF...
  • ToolGuy: U.S. sales in 2019: – Kia Telluride 58,604 – Hyundai Palisade 28,736 – Ford Explorer...

New Car Research

Get a Free Dealer Quote

Who We Are

  • Matthew Guy
  • Timothy Cain
  • Adam Tonge
  • Bozi Tatarevic
  • Chris Tonn
  • Corey Lewis
  • Mark Baruth
  • Ronnie Schreiber