The car that knows when you'll get in an accident before you do

I’m behind the wheel of the car of the future. It’s a gray Toyota Camry, but it has a camera pointed at me from the corner of the windshield recording my every eye movement, a GPS tracker, an outside-facing camera and a speed logger. It sees everything I’m doing so it can predict what I’m going to do behind the wheel seconds before I do it. So when my eyes glance to the left, it could warn me there’s a car between me and the exit I want to take.

A future version of the software will know even more about me; the grad students developing what they’ve dubbed Brains4Cars plan to let drivers connect their fitness trackers to the car. If your health tracker “knows” you haven’t gotten enough sleep, the car will be more alert to your nodding off.

“Our entire focus is to put a lot of sensors inside the car to monitor the driver, and build predictive systems,” Ashesh Jain, one of the project leads, told me. The car should “know in advance what you’re going to do.”

The team has gathered almost 1,200 miles worth of driving data on freeways and residential areas in Nevada and California, along with inside-the-car video from 10 drivers — 11, now, if you include me. They’re analyzing that video with facial recognition and eye tracking software to figure out what signals in your face correlate to certain driving maneuvers, which they can track via other sensors in the car. If you’re looking at your left rearview mirror, for instance, chances are you’re going to switch into the left lane. In a paper published this week describing the first stage of the project, the researchers report that they’ve build an algorithm capable of predicting driving maneuvers 3.5 seconds before they occur, with about 80 percent accuracy. It’s more accurate and can predict farther into the future that other experimental systems to date.

Commercially available cars already have systems that help drivers avoid accidents. Mercedes has offered “drowsiness monitoring” for years. Cars made by the German automaker and Hyundai can also slow down more quickly if it senses that the force you’ve applied to the brakes isn’t enough to avoid a rear-end collision. But Jain and his colleagues say that studying our facial expressions and glances to anticipate turning or whether you’re distracted checking your phone will help the roads be safer. Audi, Honda and Hyundai are interested in licensing the technology, but nothing’s been signed yet.

Right now, its predictions take place offline, after the driving data has been collected and analyzed, as you can see in the video above. The sedan doesn’t yet give drivers any feedback. All it does is record your face while you’re driving so the algorithms can use that to figure out what you’re going to do. But feedback is coming. In the next few weeks, the team will begin placing more sensors and feedback systems into the car to warn drivers of impending mistakes. They’ll be able to see how they react, and tweak their system accordingly to maximize the benefits.

It’s cool, but still limited. Right now, their system can make only five predictions: right turn, left turn, left lane change, right lane change, and driving straight. Driving, of course, involves many more moves, and predicting those will require more cameras and sensors, inside and outside the car, including near the floor to monitor people’s feet. If sensors and algorithms have a blind spot, that could result in the very accidents they’re meant to help avoid.

Unlike in self-driving cars, people won’t be passive passengers; they need to remain engaged. That means good alert systems. Brad Stertz, a PR rep from Audi, told me the company was cooking up a traffic jam assist system that lets a driver disengage from driving when a car is going slower than 40 mph on the highway. When the flow of traffic speeds up, it would alert a driver that she needs to start driving again. The company anticipates technologies like that to be in commercial cars around 2018, assuming car manufacturers and legislators can come to an agreement of when this type of technology is safe to use.

All these sensors in the car, of course, mean that new kinds of data about drivers is being generated.Car insurance companies already put black boxes in cars and give discounts to good drivers; this is an even bigger black box. Jain and his colleagues have already discussed a business model that involves letting consumers share their data with insurance companies in exchange for better rates.

After I left Stanford, I couldn’t help but feel excited about a car that would help me be a better driver. I’d be a liar if I said I’ve never almost bumped into a car switching lanes or had to jolt myself awake. I would welcome a system that could help me avoid that. But there’s also a huge cost involved: my data, and with it, my privacy. One of the few things I love about driving is the idea that my car is my own private bubble. With technologies like these, that is going to quickly become a thing of the past, as prying computerized eyes track us incessantly, whether we realize it or not.

Daniela Hernandez is a senior writer at Fusion. She likes science, robots, pugs, and coffee.