AI/ML: What Really Matters for Computer Vision in Self-Driving Vehicles

I recently had an engaging conversation about computer vision in self-driving vehicles with some participants of the technology leadership course I teach*. Our discussion centered around an article that labeled LiDAR as a "shortcut technology," suggesting it is good for demonstrating self-driving capabilities but not viable for the long-term. This sparked a lively debate. Some attendees defended the utility and superiority of LiDAR, particularly considering the latest innovation in optical 3D sensors.

Both viewpoints have merit, and the divergence in opinions underscores a fundamental concept in AI: it’s not the technology that matters most, but solving a customer problem. Technology enables the provision of actionable intelligence, whether autonomously or through human interaction, and this capability addresses the customer problem, which should be the primary objective in any AI-driven application.

Beyond Seeing: Understanding the Environment

LiDAR (Light Detection and Ranging) technology has seen significant advancements in recent years. It offers a quick and accurate 3D representation of the surroundings, enabling a vehicle to navigate efficiently. This capability is impressive but only a part of the equation. While LiDAR allows a machine to “see”, it does not help it “understand” its surroundings. Understanding the environment and being able to provide actionable intelligence is crucial for making safe autonomous decisions.

For a vehicle to drive autonomously, it needs to see, comprehend what it sees, understand how it fits in the current environment, and then determine its future interaction with the environment. For instance, recognizing a pedestrian and avoiding a collision, identifying a small piece of paper and driving over it, or detecting a nail and steering clear of it—these actions require understanding, not just vision. This distinction is highlighted by the recent Cruise accident in San Francisco, where a lack of semantic and situational understanding led to a tragic fatality.

Semantics matter. Understanding the meaning of objects in the environment enables a machine to decide how to act, typically through a combination of AI/ML and deterministic algorithms. The primary objective is to drive the vehicle safely. Technologies that help the machine "see" are essential but not sufficient to provide the final solution.

Vision Technologies Are Not Mutually Exclusive

Combining signals from LiDAR, cameras, and radar can significantly enhance safety. However, implementing all these technologies together incurs costs in terms of material expenses to install on a vehicle, increased opportunities for system failures, and the complexity of training models to integrate all the signals. Finding the perfect balance is a challenging operational goal. It involves making efficient use of all collected information to ensure the vehicle operates safely while avoiding expensive solutions that generate excessive, unusable data. This balance requires a nuanced understanding of the strengths and limitations of each technology (e.g., LiDAR can be unreliable in foggy, rainy, or snowy weather) and how they can work together to create a comprehensive and reliable autonomous driving system.

Conclusion

The debate over LiDAR's long-term value highlights a critical point: technologies that enable a machine to see—or, more broadly, to perform a task—are just enablers, not the ultimate solution. What truly matters is the machine's ability to understand and act on the information gathered by these technologies, ensuring safe and effective navigation.

This principle applies to all AI applications, not just self-driving vehicles. As we continue to innovate and refine AI technologies, maintaining a focus on solving customer problems will be crucial for driving the successful development of truly valuable AI-enabled products.

?

*The Technology Leadership Program at UC Berkeley-Haas is an executive education course that covers AI/ML, digital transformation, and fintech, with the addition of live seminars on leadership and how to drive transformation in complex organizations.

?

References

https://discuss.luxonis.com/blog/701-lidar-vs-monocular-depth-vs-stereo-depth-in-autonomous-driving

https://lumotive.com/technology/

Mariella Dreissig, Dominik Scheuble, Florian Piewak, and Joschka Boedecker, “Survey on LiDAR Perception in Adverse Weather Conditions”, 2023

Jiyoon Kim, Bum-jin Park, and Jisoo Kim, “Empirical Analysis of Autonomous Vehicle’s LiDAR Detection Performance Degradation for Actual Road Driving in Rain and Fog”, 2023

Arvind Srivastav and Soumyajit Mandal, “Radars for Autonomous Driving: A Review of Deep Learning Methods and Challenges”, 2023

A. Howard, L. H. Matthies, A. Huertas, M. Bajracharya and A. Rankin, “Detecting Pedestrians with Stereo Vision: Safe Operation of Autonomous Ground Vehicles in Dynamic Environments”

Lorenzo Bertoni, Sven Kreiss, Taylor Mordan, and Alexandre Alahi, “MonStereo: When Monocular and Stereo Meet at the Tail of 3D Human Localization”, 2021

Wenyu Chen, Peixuan Li, and Huaici Zhao, “MSL3D: 3D object detection from monocular, stereo and point cloud for autonomous driving”, 2022