show us the data —

This startup’s CEO wants to open-source self-driving car safety testing

Post Uber death, robo-taxi startup will share safety documents, processes, and code.

A lidar scanner

Uber's fatal car crash last month continues to have repercussions, with self-driving taxi startup Voyage announcing today that it will open-source its safety procedures, documents, and code in the hope of avoiding future deaths.

"I had to spend time after [the Uber crash] calming people down, telling folks at our deployments that it was an isolated incident," says Voyage CEO Oliver Cameron in an exclusive interview with Ars Technica. "But the truth is that everyone in the industry is reinventing the technology and safety processes themselves, which is incredibly dangerous. Open source means more eyes, more diversity, and more feedback."

Starting today, Voyage will begin to share safety requirements, test scenarios, metrics, tools, and code that it has developed for its own Level 4 self-driving taxis. Five Voyage cars are currently deployed carrying passengers within two retirement communities in California and Florida.

The initial release, which Voyage calls Open Autonomous Safety (OAS), will take the form of a GitHub repository containing documents and code. The functional safety requirements are Voyage's interpretation of the ISO 26262 standard for automotive safety, updated for autonomous vehicles. "This is our internal driving test for any particular software build," says Cameron. "It lets us evaluate our designs and look for the different ways they can fail in the real world."

Stress-testing self-driving cars

The functional testing material has scenarios designed to stress-test cars in simulations and on streets. Voyage has developed step-by-step scenarios that detail how its cars should respond to hundreds of situations, primarily focused on suburban environments. Among them is the one that Uber failed when its car ran down Elaine Herzberg in March: a pedestrian jaywalking across a divided road.

"Having a template of what your vehicle should be doing in these situations helps inform how you build your technology," says Cameron. "But it takes a hell of a lot of time to think through all the scenarios and to write the software to the right level of quality. I hope the community adds scenarios for environments we don't yet support, such as high-speed freeways."

Because OAS scenarios are written from the viewpoint of how a car should ultimately behave, they should be applicable regardless of the vehicle being developed or the technology it uses, says Cameron. However, Voyage is also sharing its fault-injection tests—code written for specific components to simulate errors or damage that might take years to show up in reality.

"If you want to replicate taking a baseball bat to your $85,000 lidar, you probably don't want to actually take a baseball bat to your $85,000 lidar," says Cameron. "But you can model what that would look like with fault-injection testing to get to a better place."

Although Voyage's code will only work with specific sensors, many are common devices (such as Velodyne lidars and Bosch ECUs) that other self-driving companies also use. Voyage will also make its training curriculum and safety-driver handbooks available.

"Overall, we think it's a pretty comprehensive set of materials," says Cameron. "We hope that the industry adopts it as a standard or, at the very least, that a few companies will use it and hopefully prevent another Uber incident from occurring."

Being open to safety

This is not the self-driving car industry's—or even Cameron's—first shot at open sourcing. Udacity, the online learning company that Voyage spun out from, is building its own open source self-driving car. Another start-up, Comma.ai, has also released the software for its aftermarket autonomous car kit, following a tangle with regulators. And Chinese search giant Baidu is slowly developing an open source platform called Apollo that it hopes will become the Android of self-driving cars.

Cameron believes that OAS' focus on safety and testing, rather than driving features, will smooth its adoption in a highly competitive industry. One aspect of the release is likely to be controversial, though. Cameron wants companies to standardize and adopt Voyage's metrics for the performance of self-driving cars based on the percentage of any particular trip that is handled by human or robotic systems.

He admits this could prove an uphill struggle. "Everyone is giving different metrics to different people," says Cameron. "Savvy VCs these days know that disengagement rates [how often a car hands control back to a human operator] can be entirely flawed. They'll ask you, hey, what's your real disengagement rate? We know the number you gave to the California Department of Motor Vehicles; now give us your real one."

Jon How, professor of astronautics at MIT, says, "Creating an open source library that enables teams to collaborate on developing even better solutions is a great idea, provided there is some oversight to ensure quality control. Hopefully, their decision might motivate others to also release their tools and test approaches."

Sam Lauzon is an engineer at the University of Michigan's Transportation Research institute, currently developing his own open source automotive cybersecurity software called Uptane. "An open source library can be fantastic for companies to lower implementation time and costs, but at the same time, they're inheriting any bugs or flaws that go along with the distribution," he says.

Cameron agrees that the current OAS release will be rough around the edges: "Like any open source project, it won't be perfect at the beginning. But it does mean that everyone won't have to keep reinventing the wheel, at least from a safety perspective."

Channel Ars Technica