Car makers can’t “drive their way to safety” with self-driving cars

It’s a matter of statistics and confidence levels.

Credit: Ford

The push for self-driving cars—at least here in the US—is happening mostly in the name of increasing road safety. More than 33,000 people die on US roads each year, and the National Highway Traffic Safety Administration says its data shows that “in an estimated 94 percent of crashes, the critical cause is a human factor.”

Advanced driver-assistance systems (think Tesla’s autopilot or the semiautonomous mode on Audi’s A4) are already a boon to drivers, reducing fatigue and keeping an ever-vigilant watch out for hazards, but the RAND Corporation has just published a study that suggests we may never be able to prove the safety of a self-driving car.

“Under even aggressive testing assumptions,” the authors write, “existing fleets would take tens and sometimes hundreds of years to drive these miles—an impossible proposition if the aim is to demonstrate their performance prior to releasing them on the roads for consumer use. These results demonstrate that developers of this technology and third-party testers cannot simply drive their way to safety. Instead, they will need to develop innovative methods of demonstrating safety and reliability.”

At issue is how much confidence one has in the statistics. Fatal crashes are rare events—in 2014, there were 1.08 deaths per 100 million vehicle miles. Proving an autonomous car is at least that safe, according to RAND, would take quite a while. “To demonstrate that fully autonomous vehicles have a fatality rate of 1.09 fatalities per 100 million miles (R=99.9999989%) with a C=95% confidence level, the vehicles would have to be driven 275 million failure-free miles,” RAND says. “With a fleet of 100 autonomous vehicles being test-driven 24 hours a day, 365 days a year at an average speed of 25 miles per hour, this would take about 12.5 years.” (RAND uses a figure of 1.09 deaths per 100 million miles.)

RAND says that even simulations and virtual testing may not be enough to prove autonomous cars safe, posing a challenge for policy makers, insurers, and the car industry. At a recent discussion on regulating autonomous cars, state and federal regulators both expressed their concern over getting things wrong, fearing the inevitable pillorying that will accompany the first fatal autonomous car crash.

Not everyone in the car industry is on board with autonomous driving. On Sunday, we reported that some automakers—ones that don’t appear to have a heavy research portfolio in autonomous cars—have called for the NHTSA to slow down.

But this seems unlikely. Regulators in the US want to harmonize regulation at the state level so that we don’t end up with a patchwork of rules where cars happily drive themselves up until reaching a state line where control has to be handed over to the human on board. And in Germany, Chancellor Merkel has also promised to revise laws to allow for testing autonomous vehicles.

Fully autonomous cars—what the NHTSA classifies as level 4, capable of driving from point A to point B with no human intervention—are being tested in a handful of locations in the US and elsewhere. But most industry experts Ars has spoken to think we’re still more than a decade away from cars that can cope with Manhattan or Bangkok during rush hour.