Ferrari Patents Augmented Reality System Designed to Merge Real and Virtual Racing

Ferrari has filed a patent with the United States Patent and Trademark Office that leverages augmented reality to allow real-world racing to crossover into the virtual realm. By using wireless networks and the real vehicle’s positional data, Ferrari thinks it can replicate it on screens at home with a high degree of accuracy.

But the really slick stuff comes into play once that experience is replicated the other way round. The special interface device the automaker has filed its patent for also replicates virtual vehicles competing against it via an advanced head-up display (HUD) and some feedback features made possible by electronic controls.

With modern vehicles lacking direct, physical connections between controls and the relevant systems, Ferrari can have real vehicles bumping into virtual ones and responding accordingly. All the individual components required to make something like this possible already exist. The company would simply need to develop the necessary software allowing the above to happen in a manner that wouldn’t be unsafe.

While electronic controls do offer simplified vehicle architectures, adaptive levels of performance, and novel concepts like what we’re talking about here, serious drivers have bemoaned the lack of feedback they offer. Electronic brakes have to simulate pedal feel, making it difficult to anticipate brake fade. Electronic steering has to simulate resistance and feedback from the road, making it harder to understand how the vehicle is actually responding to your inputs. But they’re absolutely necessary if you want to create a virtual analog of a real-world vehicle.

Though just knowing that connected cars can be controlled in this manner does open up a lot of questions about security. Considering that Ferrari’s idea would also allow the computer system to “bump” a physical vehicle based on virtual inputs from someone operating remotely, one wonders how that could be misused by bad actors.

Based on reporting from CarBuzz, Ferrari hasn’t yet reached the point where it will need to address those issues. For now, it just wants to ensure it owns the rights to the relevant concept.

From CarBuzz:

Whether via a 3D model or render, the driver of the road car will be able to see a projection of the virtual car as if it were on the track alongside them. The design goes so far as to project the virtual vehicle in the road car's wing mirror if its position is behind the road car.

If contact occurs between the road car and the virtual car, the driver of the road car will sense this via haptic feedback in the seat or steering wheel. Depending on where the contact takes place, the driver of the real car will even feel a deflection in the vehicle's trajectory, as if actual physical contact between two cars has occurred.

The real-time data transfer will likely be one of the greatest challenges for this invention to work, with Ferrari mentioning 5G and even 6G connectivity and a latency of under 500 milliseconds.

The competing driver sitting in front of their gaming console will perceive the real car as a virtual competitor, even though it's a real car being driven in real time. Although it all sounds quite complex, the basic technologies are already in place to make such a racing method plausible, and other manufacturers have also dabbled in virtual reality high-performance driving.

It’s not a terrible idea. BMW floated a similar concept with its ///M Mixed Reality that uses goggles to add virtual obstacles and imagery to people driving on real tracks. Audi and Porsche (along with a few other brands) have also toyed with using vehicle controls and headsets for racing games to be played inside a stationary vehicle.

Ferrari’s concept just takes things a bit further and presumably dumps the clunky headgear. It’s hoping to leverage the tech to enhance the ownership experience and bolster its community. The company notes in its patent filing that most of its customers aren’t professional racing drivers and aren’t likely to qualify for a lot of events or even have the time to schedule regular track days.

This limits competition, which Ferrari believes is important. But the patent is supposed to create opportunities for friends to race together more often while also giving everyone a little extra practice.

It’s not uncommon for driving enthusiasts to overestimate their skill level behind the wheel, especially on vehicles they have limited experience with. We want to believe climbing into the cockpit of an absolute monster of a car will translate into a driving display so good that onlookers cannot help but weep. But the reality is that you need to familiarize yourself with the vehicle as much as the roads you’ll be driving on.

Unfortunately, that can be exceedingly difficult when the car in question boasts an extremely high performance threshold that can’t be touched anywhere outside of a racetrack. Ferrari knows this and is hoping to create more opportunities to give its customers and their friends those kinds of experiences.

It’s something that certainly would have seemed like a stupid gimmick a decade earlier. However, we know that modern racing simulators have reached a point where there is some very real skill overlap taking place. It’s literally the plot of the 2023 Gran Turismo film, which is based on a true story where Nissan used the titular racing sim to help fill the roster of its professional driving team.

What Ferrari wants is to utilize those advanced racing rigs automotive enthusiasts already have in their homes to bridge and bring them into real-world events that are happening in real time.

Assuming it works as intended, the implications of this technology could be massive. While I wouldn’t call any of the people I know with a Ferrari in the garage my friend, I could see myself strapping in at home to virtually compete in televised motorsport events and even paying extra for the privilege.

We actually kind of saw this happening when racing was stopped during the pandemic. With tracks closed, drivers used home simulators to keep their skills sharp and it wasn’t long before they were competing in events with random strangers over the internet. Motorsport organizers are likewise constantly seeking new ways of engaging the audience and the above seems like an idyllic solution — especially if they’re hoping to snag younger people.

It’ll be exciting to see if this patent filing turns into something tangible. While all the individual components seem to be there, it’s often easier said than done to bring it all together. However, Ferrari has some very exciting ideas that could fundamentally change how we experience motorsport.

[Images: USPTO; Ferrari]

Become a TTAC insider. Get the latest news, features, TTAC takes, and everything else that gets to the truth about cars first by  subscribing to our newsletter.