Inside Cruise’s Bumpy Ride: The Limits of Self-Driving Cars

It also means that GM will find it difficult to achieve its aim of launching by next year a taxi service using self-driving cars “at scale in dense urban environments.”

The problem is the nature of the technology, which requires cars to follow traffic laws exactly—and handle countless individual scenarios. Some of those can confuse the vehicles. Cruise cars frequently swerve and hesitate, according to interviews with people who have knowledge about the program and internal data about its challenges. They sometimes slow down or stop if they see a bush on the side of a street or a lane-dividing pole, mistaking it for an object in their path. This overly cautious behavior creates frequent problems for drivers in regular cars, in some cases causing accidents—even if the human driver is legally at fault.

Cruise, alongside Alphabet’s Waymo, has garnered outsize attention in the race to build self-driving cars. Waymo, which has said it has spent far more than $1 billion on research and development, decided to focus on the relatively easier suburban areas around Phoenix in order to be first to launch a fully autonomous ride-hailing service to the public.

The full extent of the Cruise vehicles’ limitations (scroll down to see a list) haven’t been previously reported. They provide a window of how daunting it is to develop a fully self-driving vehicle and why expectations about its arrival and life-transforming abilities should be curbed significantly.

Waymo and other autonomous vehicle developers have had many of the same issues as Cruise, though Cruise is one of the only programs to seriously tackle a big city. And even in less intense environments, such as suburban Phoenix, Waymo has faced plenty of technical challenges, including making “unprotected” left turns. In prior years Waymo also faced issues with rear-endings caused by its cars’ overly cautious approach.

Cruise is a “bellwether for the other big programs that are just starting to tackle the urban environment,” said Anne Widera, a former manager at Waymo and Uber’s self-driving car team who now is a consultant.

Testing Grounds

Cruise may not even be able to handle a full-fledged robo-taxi service in San Francisco by next year, as it has said it would, despite all the testing it has done in the city. Right now the fleet operates primarily in one quarter of the city, with a lot of limitations. Cruise plans to start testing next in New York City. But the driving customs and streets are so different that a lot of what the cars have learned won’t necessarily carry over.

Meanwhile, Cruise has pulled back sharply on its testing in suburban Phoenix, where rivals like Waymo and Uber have been ramping up. That way Cruise can focus on markets with better business potential. For instance, it has been eyeing Seattle as a testing ground.

The challenges aren’t lost on Cruise’s leaders and the man who oversees the unit, GM President Dan Ammann. Mr. Ammann has said many people in the tech industry underestimate the scope of the autonomous driving challenge. Because GM is tackling the toughest environments, it will be difficult for competitors to catch up, he has predicted. He has also said the company won’t release technology to the public until it is incredibly safe.

Part of the problem is human behavior: Humans break traffic laws all the time, both as drivers and pedestrians, in part because they don’t actually know the rules. They tailgate, jaywalk, roll through stop signs and get distracted by their phones. They drive in and out of gas stations with reckless abandon. That’s a big reason why self-driving cars are difficult to make work in the real world.

For their part, drivers in San Francisco who see a Cruise car on the road often are alarmed by the car’s unexpected behavior, which leads to accidents, according to people with knowledge of some of the incidents.

Last September, for example, a Ford Explorer rear-ended a Cruise car that stopped as it was turning into an intersection “to let a pedestrian clear the crosswalk,” according to an accident report Cruise filed with state regulators. Someone who was briefed about the incident said the person sitting behind the wheel of the Cruise car had to take over after sensing that the car might not yield to the pedestrian, hence the sudden stop.

In another case from last February, a Cruise car paused in front of an empty crosswalk because yellow lights were flashing on either side of it. The car’s unexpected stop caused a Toyota following the Cruise car to brake. That in turn led to a Subaru rear-ending the Toyota, which then bumped into the Cruise car.

A more recent example, from December of last year, involves an “aborted” lane change by a Cruise car. Lane changes are a longtime area of concern at the company—they’re hard for human drivers, too. The Cruise car can detect whether there is enough space for it to squeeze between two moving cars and, if there is, move toward it. And by now it has been programmed to try to give “cues” to other drivers that it is about to make that lane change, such as hugging the side of the lane that’s closest to the open space.

But problems emerge when that space suddenly shrinks. That occurs when vehicles on either end of the space change their speeds, said people familiar with the vehicles. The Cruise car doesn’t yet anticipate that when it starts entering a new lane, the vehicle behind it in the new lane will inevitably slow down in order to not strike the incoming Cruise car. Instead, the Cruise car may abort the lane-change after it begins making one. The Cruise car also struggles to complete a lane change if the car ahead of it in the new lane slows down.

(Don’t) Stay in Your Lane

That’s what happened on Dec. 7, according to a filing by Cruise with state regulators and a person familiar with the details of the incident. A Cruise car was driving in the center lane of a three-lane street in San Francisco. After the Cruise car began to merge into the left lane behind a minivan, the minivan began to decelerate and the Cruise car aborted the lane change halfway through. The Cruise car moved back into the center lane.

But a motorcycle was behind the Cruise car and was in the process of passing it when the Cruise vehicle suddenly moved back into the center lane, according to a person familiar with details of the incident. The Cruise vehicle, driving at 12 mph, wasn’t able to prevent a collision with the motorcycle, which was driving at 17 mph. The motorcycle fell over.

The motorcyclist walked away from the accident but has since sued Cruise, based on alleged physical injuries he sustained, arguing that the Cruise car “suddenly veered” back into the center lane. Cruise points to the traffic collision report filed by police after the accident, which determined the motorcyclist was at fault for attempting to overtake and pass the Cruise car on the right before it was safe to do so.

Nevertheless, Cruise executives were concerned about the situation and what led to it. The “safety driver” sitting behind the wheel of that Cruise vehicle, whose job it is to take control of the wheel if they sense a potentially dangerous situation, no longer works for Cruise, according to a person who was briefed about the incident.

After the accident, the Cruise fleet was grounded for at least several hours as engineers looked into the matter. Cruise engineers later directed the company’s safety drivers that if the car tries to abort a lane change, they should take over the wheel to complete the lane change, this person said.

Cruise spokesman Ray Wert did not discuss the incidents as well as the aborted lane change, citing the litigation. “We are operating with an abundance of caution, as this is a new technology, much like a cautious driver. As we get closer and closer to launch, we’ll be able to tune our testing behaviors, but for now we are focused on being great partners with San Francisco.”

It’s hard to know how safe these vehicles are today. Cruise and other developers have filed reports with California regulators that say that their vehicles rarely require people to take over the wheel for safety reasons, such as avoiding a collision. Cruise, for instance, said that in the 12 months ended in November of last year, its drivers took over the wheel less than once every 1,000 miles the cars drove, down from 18.5 times per 1,000 miles driven in the prior 12 months. That data placed Cruise behind only Waymo in terms of disengagement rates.

However, as The Information has reported, companies have a lot of leeway in determining what to disclose. When it comes to such reports, “I wouldn’t pay attention to any of it,” said one person who has worked at Cruise and is familiar with its accidents and close calls.

What’s Working

Cruise’s cars have come a long way since GM acquired it two years ago: Driving in the rain is easier now to the point where only heavy downpours cause the fleet to be grounded; a year or so ago, even moderate rains might have caused such a grounding.

The cars can stick to the center of their lanes and drift to one side or the other if a human-driven car or bus in an adjacent lane gets too close. And double-parked vehicles are less of a roadblock than they used to be. If the cars temporarily lose traction when they hit cable-car tracks, it doesn’t affect their performance.

Other improvements: The cars can sometimes glide through an intersection even after the traffic light has turned yellow rather than come to a halting stop. While making right turns, the cars have gotten a lot better at not cutting off pedestrians walking in a crosswalk.

And Cruise has developed an algorithm that can figure out how far it can “creep” into the middle of many intersections before trying to make a left turn. (Completing the turn successfully is another matter.) Still, that’s just scratching the surface of what a machine should be able to handle in order to replace humans.

Offering autonomous rides to customers will be a long road. Programming a car to drive as well as an average human driver—which may or may not satisfy regulators, let alone customers—means writing code to handle a seemingly limitless array of “corner” cases, or problems that come up infrequently but could lead to an accident if the vehicle isn’t prepared. This technology could take many years to get to a human level of performance.

Cruise is a “bellwether for the other big programs that are just starting to tackle the urban environment.”

Some practitioners in the autonomous vehicle field believe that without further research breakthroughs in artificial intelligence beyond what today’s prototypes are benefitting from, the cars might not be able to match human-level performance.

In the next year or so, some routes in San Francisco that have relatively few potential hazards might be ready for robo-taxis to handle some paying customers or at least hand-selected “beta testers,” like what Waymo has done in suburban Phoenix. But to turn such routes into a real business, Cruise may have to operate in partnership with a broader ride-hailing service like Lyft and Uber, whose human drivers can navigate anywhere. Customers of Lyft and Uber could be assigned to ride in Cruise cars when their routes match where Cruise can operate.

Cruise would prefer not to rely on those other network operators. That’s because the operators would take a cut of revenue. Cruise CEO Kyle Vogt has said that business from robotaxis could end up being 700 times bigger than Uber and Lyft today. But such a service, which GM hopes will help transform the company into a fleet operator with greater revenue potential than its core business, is a gleam in its eye.

The Blacklist

Cruise’s prototypes are somewhat limited geographically. Currently they only operate en masse in the northeastern quadrant of San Francisco, or about 20 square miles, including the Mission, Castro, Financial, SoMa and Marina districts, according to a person with knowledge of the area. It’s the most complicated quadrant to drive because it has the most traffic and daytime population, so it makes sense to nail that down first.

While Cruise’s employees can hail a car within that area through a special mobile app, some of them have opted not to take rides because such trips can take much longer than being driven by a human driver, sometimes by 10 or 20 minutes, this person said. The company wants to expand its app’s territory to the northwestern quadrant of the city later this year.

Part of the reason for the painfully slow trips is that some San Francisco intersections and streets are “blacklisted,” in some cases temporarily, and the cars must take circuitous routes around them. Blacklisting may occur if an intersection has a traffic light that is too faint; if there is a roundabout or if two lanes merge into one. Cruise isn’t yet trying to broadly test the cars in roundabouts or when lanes merge in a particular way.

The cars also have trouble with construction zones and can’t easily handle two-way residential streets that only have room for one car to pass at a time. That’s because Cruise cars treat the street as one lane and always prefer to be in the center of a lane, and oncoming traffic causes the cars to stop.

Radars that stick out from the side of the Cruise cars sometimes get dinged as cars try to squeeze around them from behind. These days, Cruise cars generally don’t pull over to a curb, between parked cars, to pick up or drop off passengers, possibly because there are many areas where it isn’t legal to do so.

Another issue: The cars’ sensors aren’t easily able to distinguish between bicycles and motorcycles, which makes it harder to predict the behavior of each. Tunnels are no-go zones at the moment, likely because the cars can’t connect to GPS and “localize” themselves on Cruise’s virtual map, but the cars can traverse bridges like the ones near the local baseball stadium. The cars also haven’t been broadly tested U-turns in the city for reasons that aren’t clear.

Trimming Bushes

Even stationary objects like bushes and lane-separating poles are problems. Before showing off the cars to some journalists last fall, Cruise trimmed at least one bush in the Dogpatch neighborhood so that the cars wouldn’t view it as a possible moving object and swerve unnecessarily, said a person briefed about the matter. (A self-driving car software startup called Voyage did the same thing to help its robo-taxis near San Jose.)

Mr. Vogt has insisted that teaching cars to drive in San Francisco, one of the most challenging urban environments in the U.S., is better than focusing on calmer suburban areas the way Waymo and Uber have done in Phoenix. Waymo said last fall it removed the “safety drivers” who sit behind the wheel of self-driving car prototypes to take over in case the cars run into trouble, though the cars can be closely monitored and operated remotely if they get stuck. Removing safety drivers is a step toward offering a ride that could cost less than taking an Uber or, eventually, owning a car. Cruise hasn’t said when it expects to remove safety drivers, but it probably won’t be soon.

Despite the easier environment in Phoenix, Cruise recently pulled back on its operations there. The precise reason for the pull-back couldn’t be learned, but San Francisco has a thriving ride-hailing business dominated by Uber and Lyft that Cruise could try to eat into, whereas suburban Phoenix would have less consumer demand in the near term.

Also, there are enough problems left to solve in San Francisco. It makes little sense to expand in a big way to other locales where everything is different: driving behavior, street and lane markings, traffic lights and intersections. “They’re fine-tuning the software for San Francisco,” said a person briefed about Cruise’s work. That makes it hard to replicate the experience elsewhere.

Robot Limit

The overall challenge Cruise faces is the natural result of its engineering methods. Most autonomous vehicle researchers, including those at Cruise, work with two key technologies: robotics, in which programmers write rules for the cars to follow; and deep neural networks, which are a form of machine learning known as deep learning. Such networks can process data from a Cruise car’s sensors to recognize objects around them, such as stop signs and other cars, and to detect traffic lights.

Cruise’s leaders have implied they primarily follow the traditional robotics approach in programming the car to make decisions about individual traffic scenarios, one by one, that they think it will encounter. And for the most part, decisions about where and how the car will drive are binary: Is there open space for the car to move into or not? Is there any chance, even a slim one, that an object such as a bicycle will end up in the car’s path?

But this type of “thinking” can be paralyzing, causing cars to hesitate or get stuck. As a result, the company has discussed ways to train the car to make a decision without 100% certainty—in other words, to act more like a human driver, said a person briefed about the effort. Still, the impact of such a change may be limited because the autonomous driving system can only reassess the situation around it every fraction of a second, as it tries to predict what various objects will do. That speed is many times slower than what a human can do.

“A robotics approach is great for being safe and ensuring you aren’t going to hurt anybody, and it works great in suburban areas where there are relatively few objects to predict,” said Ms. Widera, the former Waymo manager.

But “in a city, you have pedestrians, bicycles, scooters and motorcycles, often in the same field of view, all with different models and profiles. It becomes computationally costly to precisely predict where everything is going to go, and there's a chance you end up hitting a limit on computing power you can realistically and cost effectively put on the vehicle," she said.

Some autonomous driving developers such as Tesla take a polar opposite view to engineering. They believe that by gathering enough camera and radar data from cars, a deep learning algorithm can “learn” the right way to drive safely. Traditionalists view that as a fool’s errand, in part because it might require too much data to develop a system that could handle corner cases. They also say that because of the nature of deep learning systems, it would be too difficult to deduce the precise cause of any mistakes the cars make.

The jury is out on which is the right technical approach. An Uber researcher recently said both types have significant flaws. That lends credence to the idea that the race to develop autonomous cars is still wide open.

‘90-90’ Rule

Cruise, like many other developers, is trying to follow Google. Google began its self-driving car project in 2009, and by 2010 had successfully driven autonomously through 10 separate 100-mile loops in California, across urban streets and highways. In early 2017, Google spun out the venture as Waymo, a separate unit of Google owner Alphabet, after making slow progress in commercializing the technology.

Waymo has an explanation for that: “When you’re 90% done you still have 90% to go,” said Sacha Arnoud, a director of engineering at Waymo, in a recent talk at the Massachusetts Institute of Technology. In other words, writing the first 90% of the code that’s necessary for self-driving cars to function in the real world takes just 10% of the time. Completing the job, or the last 10%, requires 10 times the initial effort.

“You need to 10x the capabilities of your technology. You need to 10x your team size, including finding effective ways for more engineers and more researchers to collaborate together. You need to 10x the capabilities of your sensors. You need to 10x the fundamentally the overall quality of the system, and your testing practices,” he said.

GM set up Cruise with its electric Chevy Bolts as the vehicles for its self-driving software. Earlier prototypes of self-driving Bolts were manually retrofitted with sensors, which meant each car’s sensors were set up a little differently, leaving each car with a different personality, according to a consultant familiar with the vehicles. The latest cars, which shipped in the fall of last year, came off the company’s Michigan assembly line with the sensors already installed.

“Now they all equally suck,” this person joked.