This article first appeared in The Edge Malaysia Weekly on October 30, 2023 - November 5, 2023

FOR 10 weeks now, San Francisco has been a testing ground for robotaxis — driverless ride-hailing vehicles that take you from one part of the city to another, the way cars operated by Grab or GoTo Gojek Tokopedia in Southeast Asia do with human drivers. Though Phoenix, Arizona, has had paid robotaxis operating for almost a year, the arrival of driverless cabs in the tech capital of the world was seen as the ultimate test for the robots that drive you around town.

Robotaxis have been talked about for almost a decade now. Tesla Inc CEO Elon Musk first mentioned “converting” a huge fleet of his pioneering electric cars with full self-driving (FSD) capability “soon” way back in 2015. In 2018, he was quoted as saying that within a year there would be a million Teslas that could be used as robotaxis. Ironically, Google subsidiary Waymo and General Motors subsidiary Cruise beat him to it. 

The key to robotaxis is collecting a ton of data. Artificial intelligence (AI) then uses the data — images, videos, audio and text — to train machine learning models. Robotaxis start with human drivers and once they have collected sufficient data, AI uses that training data to teach them to drive around by themselves. 

In August, the California Public Utilities Commission announced that Waymo and Cruise were ready for prime time and allowed robotaxis to operate round-the-clock paid services in San Francisco. Last month, Austin, Texas, became the third city in the US with a commercial robotaxi operation. As of last week, Cruise and Waymo had announced they would expand into 14 other US cities including San Diego and Los Angeles, California; Miami, Florida; Atlanta, Georgia; and Dallas and Houston, Texas. Most of these cities have warm weather and flatter surfaces. In northeast US, there is snow in winter as well as more challenging terrain. Phoenix and Austin are more planned cities and therefore among the first on the list of places robotaxi operators want to be.

Initially, there were protests from urban groups after robotaxis suddenly stopped at intersections, interrupting the normal flow of traffic. On one occasion, a robotaxi collided with a fire truck on its way to an emergency scene. In a couple of cases, robotaxis could not recognise yellow-and-black caution tape that was put up to cordon off an area. They just kept driving through the tape. Robotaxis are smart enough, however, to recognise bright orange plastic cones. So protestors like anti-robotaxi group Safe Street Rebel launched a campaign to ambush and incapacitate them by placing traffic cones on their bonnets.

The cameras would recognise the cone and the robotaxi would abruptly stop in the middle of the road. There are plenty of hilarious robotaxi cone videos on YouTube.

On Oct 24, California’s Department of Motor Vehicles (DMV) told Cruise to “immediately remove all its driverless vehicles from the roads”, saying it had determined that robo­taxis were causing an “unreasonable risk” to public safety. DMV alleged that Cruise had misrepresented “information related to safety of the autonomous technology”.

The ban is clearly a huge setback for Cruise, and its main shareholder, General Motors, which has been struggling to transition from a maker of internal combustion engine-based vehicles to pure battery electric vehicles (EVs) at a time when unions are on strike demanding higher wages. It is also a setback for autonomous vehicles in general, and robotaxis in particular.

The move comes months after state regulators began investigating a spate of “concerning incidents” involving Cruise vehicles in San Francisco and ordered it to cut its robotaxi fleet in the city by 50%. It also comes just weeks after a jaywalker was struck by a car driven by a human and then flung into the path of a Cruise, which aggressively applied its brakes only to roll over the woman, dragging her for about 6m before stopping. “DMV has provided Cruise with the steps needed to apply to reinstate its suspended permits,” the agency said.

In the first few weeks of the San Francisco robotaxi experiment, a few thousand people were able to download the apps of Waymo or Cruise, just as you might download a Grab or GoJek app in Southeast Asia and order a ride. Waymo was operating about 250 cars while Cruise had cut its service to just 50 vehicles during the day and 150 at night before its Oct 24 suspension.

When I wanted to ride a Waymo robotaxi three weeks ago, I realised there was a long “wait list”. So I tried to get in touch with Waymo to let me ride because I am a journalist who wanted to write about it. Then last week I learnt that Waymo would soon increase the number of robotaxis and the wait list would soon be abolished and robotaxis would become almost as ubiquitous as Uber and Lyft.

I am a tech columnist, so it is not surprising that I am an early adopter and a big believer in technologies like autonomous vehicles. I think robotaxis can transform society in our lifetimes. I had originally planned to write this piece two weeks from now after my trip to Las Vegas to see the Sphere — a huge new spherical-shaped entertainment venue for concerts and immersive shows — where Irish rock band U2 is playing. (Google “Sphere Las Vegas” to see how spectacular it is.) Then Cruise got banned and now there is talk that DMV is taking a hard look at Waymo as well, and other cities are thinking about putting their own driverless taxi experiments on ice.

Here’s why we need robotaxis: The World Health Organization estimates that every year around the world, 1.35 million deaths are caused by traffic accidents. Over 93% of those deaths are due to human error. Consultancy McKinsey & Co estimates that for every person killed in a vehicle accident in the US, eight others are hospitalised. A total of 46,000 people died in auto-related accidents in the US last year, and 2.3 million are injured in one way or another in an auto accident in the country annually. The two main causes of auto accidents are driving under the influence of alcohol and getting distracted while driving, like talking on the phone.

An average one-way commute in some of the busiest cities in America takes about an hour and covers less than 16km. The No 1 reason why people want to work from home rather than go to the office every day is because they hate the commute. Driverless vehicles could save half or more of the commuting time in most cities. In some cases, a third or up to half of the downtown area — the most expensive real estate in the city — is earmarked for parking. A third of the traffic in America’s busiest downtowns is caused by drivers looking for parking.

Robotaxis and other driverless vehicles would free up huge amounts of space in New York, Chicago, San Francisco and other busy downtown areas, helping to make housing and real estate affordable for residents and businesses. And, oh, one other thing: Robotaxis will dramatically reduce the number of cars on the road. Studies show one robotaxi will likely replace up to five private cars. “Autonomous taxis could be the most impactful innovation in history,” says Tasha Keeney, director of investment analysis at tech-focused fund manager ARK Invest.

Here is how they work: Most autonomous vehicles, like robotaxis, have a rapidly spinning tube mounted on the roof. That’s a ­LiDAR, or Light Detection and Ranging sensor. The role of LiDAR, which works in tandem with an array of other sensors on the car, is to measure the distance of objects relative to the position of the car in three dimensions or 3D. Remote-sensing LiDAR uses pulses of ultraviolet laser lights to measure the distance, speed and altitude of physical objects, and map out the surrounding environment with impressive fidelity. Unlike other sensing technologies like, say, cameras, LiDAR’s performance is not degraded in low light conditions.

But here is the rub: LiDAR is a very expensive technology. Robotaxis currently on San Francisco streets cost over US$100,000 (RM478,808) each to make. Add in some bells and whistles as well as marketing and distribution costs and a decent mark-up for profits, and you would be looking at close to US$150,000 per car.

Tesla has taken a completely different route with its own FSD technology, which it wants to deploy for robotaxis. CEO Musk has said that “anyone relying on LiDAR is doomed”. Tesla’s AI guru Andrej Karpathy has been quoted as saying that “LiDAR is a shortcut” so Tesla wants to focus on an affordable solution that uses cameras and computer vision.

How does that work? Computer vision is a part of AI where humans teach computers to interpret and understand digital images or videos using AI based on neural networks, machine learning and deep learning methodologies. Computer vision uses images and videos to understand a real-world scene in order to create a 3D map, which enables self-driving cars to identify, classify, and detect different objects.

Essentially, the computer vision that Tesla is betting on relies on the training that was done during the data annotation process to recognise objects on the streets. It looks at each object the same way a human would and identifies them based on their specific characteristics. Musk has said his cameras and computer vision-based FSD is far superior and cheaper than LiDAR, which Waymo and other autonomous car makers are using. His vision is to deliver a robotaxi that costs under US$25,000 to buyers, or a fraction of what a Waymo robotaxi would cost if Google’s parent Alphabet were selling it today.

So, who has the edge? Tesla’s rivals like Cruise and Waymo, or indeed iPhone maker Apple Inc and e-commerce behemoth Amazon.com, which have been developing their own autonomous cars secretly for years? ARK’s Keeney argues that Tesla has more autonomous-driving data than all other car makers combined. That gives the EV pioneer an edge in developing FSD software because data is the cornerstone of AI. Tesla plans to monetise its FSD technology by selling software directly to consumers or drivers like you and me, and providing autonomous ride-hailing services with robotaxis powered by its FSD platform — in other words, become a competitor to the likes of Uber, Grab and DiDi.

The way Keeney sees it, “autonomous vehicles could be one of the most productive innovations of all time, impacting global gross domestic product by 20% over the next decade”.  The ARK analyst  estimates that net GDP gains globally could approach US$26 trillion by 2030.

The promise of autonomous vehicles is simple: They will be safer and help reduce congestion. The key problem with autonomous cars is that designing them to anticipate rare events like a speeding fire engine hurtling towards the site of a disaster through a red light, an ambulance racing a dying patient to a hospital or an idiot making a sharp left turn after giving an indicator that he was turning the other way is hard. Humans are easily flummoxed by lesser things so you can imagine how hard it is to make robotaxis understand what’s going on.

But AI can fix it over time. You need to collect a ton of training data, which then makes it easier for the autonomous car to make critical decisions. Waymo and Cruise have collected data but only in a handful of cities. Tesla has been operating across the US and around the world collecting a lot of training data that can be put to use to drive autonomous cars safely. Clearly, robotaxis have made a mess of things in San Francisco recently. But it is early days of the experiment. Robotaxis are coming to cities near you, though they won’t be ubiquitous for years.I believe legacy automakers will at some point start installing cameras in their cars, not just new ones but older ones as well and start collecting data that can be used to make autonomous cars safer on our streets.

Assif Shameen is a technology and business writer based in North America 

Save by subscribing to us for your print and/or digital copy.

P/S: The Edge is also available on Apple's App Store and Android's Google Play.