Even with auto makers and tech companies sprouting projects across the country, California remains a fertile testing ground for self-driving vehicles.
Dozens of companies are testing autonomous-driving technology on the state's roads, and they collectively drove at least 2 million miles in the most recent statistics reported to the state's Department of Motor Vehicles, which were released last week. That's more than four times the 500,000-mile total compiled during the last reporting period and perhaps a signal that, despite whatever broad industry headwinds self-driving technology has faced in recent months, research and development is continuing at an accelerated pace.
In reality, the mileage numbers likely are higher, but just 48 of the 62 companies permitted to test autonomous vehicles in California were required to file reports.
The statistics come from annual reports that permit holders are required to file with the DMV, which detail the number of miles tested in autonomous mode and the number of times drivers or systems disengaged from computer-controlled driving or that a human safety driver intervened to ensure smooth operation.
The reports serve as a snapshot of the individual companies more than a means for drawing conclusions about the competence of self-driving systems and the ongoing competition among some of the biggest companies in the space. Companies test in a variety of conditions—complex, urban areas and comparatively easy highways, for example—and for a variety of purposes, making it difficult to discern the value of comparisons.
"The disengagement reports now are as much a voyeuristic exercise in who is testing and how much as they are anything else," said Mike Ramsey, senior research director at Gartner, a global technology consulting firm. "The location of the testing, the thresholds for disengagements, the accuracy in reporting all could vary by participants. It is interesting for sure. ... But it's not something you could rely on in any meaningful way."
Still, the disengagement reports are the most insightful peek that outsiders get into individual testing efforts, and they can be useful in benchmarking how companies change their performance year over year.
Waymo, the company spun from Google's self-driving car project, increased its testing presence dramatically, logging more than 1.25 million miles on California's public roads in the reporting period. The company reported 114 disengagements, a rate of 1 for every 11,018 miles traveled. In the previous reporting period, the company's vehicles had traveled 352,544 miles in autonomous mode with 63 disengagements, a rate of 1 for every 5,596 miles.
Cruise Automation, a subsidiary of General Motors, has more autonomous vehicles than any of its competitors on public roads in California, with 180 registered with the DMV. Cruise cars drove 447,621 miles on public roads, with almost all of that testing occurring on city streets in San Francisco, according to a company spokesperson.
The company's cars experienced 86 disengagements, a rate of 1 for every 5,205 miles. In the previous reporting period, Cruise vehicles had 105 disengagements over 129,764 miles, a rate of 1 for every 1,236 miles.
"San Francisco is our home, because it allows us to improve more quickly," Albert Boniske, the company's vice president for product integrity, wrote in Cruise's report to the DMV. "We experience more complexity on a per-mile basis here—think pedestrians, cars, cyclists and raccoons—providing the richest data for our object detection, prediction and response functions. Not only is San Francisco difficult on its own, but we actively tailor our testing to find the hardest situations and get better at navigating them."
Apple's first report
Apple filed its first disengagement report, and, if anything, it serves as a cautionary tale for examining the ambiguities within the disengagement reports. Overall, the company reported 80,739 miles driven and 76,585 disengagements during a reporting period that stretched from April 2017 through November 2018. But the company noted that it initially separated disengagements into the categories of manual takeovers and software disengagements, and it had 40,198 of the former and 36,359 of the latter through June 2018.
Starting in July, the company shifted its parameters on how it would report disengagements.
"We began to categorize our disengagements in greater detail," Jamie Waydo, the company's senior director of autonomous systems engineering, wrote in the DMV filing. "In particular, we began to flag certain disengagements as 'important disengagements'. Important disengagements occur in situations that might have resulted in a safety-related event or a violation of the rules of the road."
Following its new methodology from July 2018 forward, Apple reported 28 "important disengagements" over 56,135 miles, a rate of one per every 2,005 miles driven.
Quiet companies
The disengagement reports also reveal which companies are no longer testing, at least in California. Tesla says it did not log any miles in autonomous mode during the reporting period, and ditto for autonomous-shuttle company Navya and Subaru. Uber's permit to test in California expired last March, and Lyft said it didn't operate any vehicles in autonomous mode during the reporting period.
If the disengagement reports offer an incomplete picture of the state of autonomous-vehicle testing, Adam Scow, a senior advocate for Consumer Watchdog, emphasizes that California's regulations nonetheless remain a vital way for consumers and government officials to avoid an informational vacuum.
Other states with autonomous- driving testing require no such disclosures.
"Besides the occasional tragedy, the public is in the dark about what's happening in other states," Scow said. "It's only because of California's rules that the public can find out what's happening when companies use public roads as their private laboratories."