The letter from the San Francisco Municipal Transportation Authority (SFMTA) to the National Highway Traffic Safety Administration (NHTSA) shines a bright spotlight on a major weakness of current automated vehicle technology – explainability. The letter is in reply to a request for comment from interested parties by NHTSA regarding General Motors’ request for exemptions from traditional safety regulations for GM’s Cruise Automation Origin vehicle to operate driverlessly on public roads.
The letter highlights the performance shortcomings of Cruise’s existing fleet of self-driving vehicles – based on Chevy Bolts – and the agency’s disappointment in Cruise’s responsiveness to multiple concerns that the agency has expressed. While the letter also highlights the clashing jurisdictions of Federal and local authorities, it also raises alarms regarding the potential negative impact that might result from Cruise unleashing hundreds or even thousands of vehicles on San Francisco streets – or, in fact, the streets of any city.
At the core of the SFMTA’s concerns though is not only Cruise’s failure to respond to questions regarding day to day operation of its vehicles or to concerns regarding particular incidents – the letter raises questions as to Cruise’s ability to explain how or why its vehicles are doing what they are doing. This breakdown reflects a shortcoming in artificial intelligence and machine learning technology where users or creators are unable to explain the output of their own algorithms.
The first evidence of this explainability breakdown emerged earlier this year when NHTSA opened investigations into phantom braking incidents plaguing vehicles from Tesla operating in Autopilot or Full-Self-Driving. Tesla vehicles are known to periodically come to a stop on highways – and the company has been unable to either explain or remedy the problem.
This experience echoed the “unintended acceleration” events that struck Toyota vehicles years ago and spurred a Congressional investigation and NHTSA’s outreach to the National Aeronautic and Space Administration (NASA) to try to explain the phenomenon. Of course, the Toyota incidents were not tied to artificial intelligence or machine learning.
The SFMTA letter cites multiple circumstances of Cruise vehicles slowing or stopping mid-block in the flow of traffic for no reason – including situations where emergency responders were impeded. The agency also expressed its unhappiness with Cruise vehicles not pulling out of traffic lanes and over to available curb space to pick up or drop off passengers – as required by law.
The agency further raised questions as to the timeliness of Cruise’s responsiveness in the event of vehicle failures. In these situations there were delays in making contact with appropriate personnel at Cruise as well as additional delays in Cruise personnel coming to rescue inoperable Cruise vehicles.
SFMTA’s concerns were elevated by its anticipation of the Bolt-based Cruise vehicles – which are equipped with steering wheels and brake and accelerator pedals – being replaced with much larger Cruise Origin vehicles that lack such manual vehicle controls. In fact, the lack of those controls are the motivation for GM to be requesting regulatory waivers for as many as 5,000 Cruise Origin AVs.
Cruise personnel can easily reposition or remove Bolt-based AVs, but Origin vehicle failures are expected to require the involvement of flatbed or tow trucks to remove or reposition the vehicles.
One of the SFMTA’s greatest concerns, though, expressed early in the letter, is the anticipated impact of steering wheel-less AVs operating in San Francisco in substantial numbers. According to the SFMTA’s own research, the introduction of a total of 5,700 Uber and Lyft vehicles six years ago was responsible for 25% of all travel delays in the city at that time.
Cruise’s current fleet operating in San Francisco without drivers currently consists of considerably less than 100 vehicles. The company has logged less than 20,000 miles of autonomous operation through May 22, according to SFMTA.
Cruise’s failures to adequately respond to local regulatory authorities and/or to explain the failure or idiosyncratic functioning of its vehicles marks an important turning point for the AV industry. SFMTA reported a significant uptick in 911 calls to emergency responders in connection with the erratic behavior or apparent failures of unmanned Cruise vehicles – even at their currently low on-road volume.
Cruise has made some efforts to reach out to the public with marketing messages and to try to explain itself, its operations and its goals. The more salient authority that is in desperate need of this kind of outreach is the SFMTA and local emergency responders who have been forced to cope with the evolving operational shortcomings of Cruise vehicles and the public’s reaction to them.
It’s worth noting that few such complaints or pushback has arisen from the operation of Waymo’s AVs. Waymo has thus far been operating with traditionally equipped and regulatory-compliant vehicles that do not require waivers from NHTSA to operate.
With its letter, the SFMTA posits a nightmare scenario where Cruise might – on its own – decide to introduce hundreds or thousands of its driverless AVs on the streets of San Francisco. The waiver request from GM to NHTSA forces the SFMTA to ponder the impact of such a prospective deployment.
For the average San Francisco native, the letter suggests that it might be time to put the brakes on all robotaxi activities until and unless the city decides that robotaxis are indeed a desired transportation objective. One hint as to the unlikeliness of this are the additional objections and concerns expressed by the SFMTA regarding accommodations for residents with disabilities. It was only after a major regulatory and legal tussle that the SFMTA was able to obtain appropriate concessions from Uber and Lyft for such residents.
In the end, Cruise needs to come clean and clean up its act. And the SFMTA has now raised questions that all municipalities must ask: Do we want robotaxis? How do we want them to operate? And how many are we prepared to accomodate?
Letter from the SFMTA: https://regmedia.co.uk/2022/09/26/letter_to_nhtsa.pdf
Also Read:
U.S. Automakers Broadening Search for Talent and R&D As Electronics Take Over Vehicles
Super Cruise Saves OnStar, Industry
The Truly Terrifying Truth about Tesla
Share this post via:
The Data Crisis is Unfolding – Are We Ready?