Autonomous Tech: The Rise of the Pedestrian

Two pedestrians – one named and one so-far unnamed - redirected the future of autonomous vehicle development in the U.S.? Elaine Herzberg lost her life in a March 18, 2018, crash with a self-driving Uber test vehicle with a human safety driver in Tempe, Ariz.? The crash ultimately led to Uber’s sale of its Advanced Technology Group to Aurora Innovation in 2020.? Robotaxi maker Cruise recalled all of its vehicles in November of last year after one of them ran over and dragged a pedestrian in San Francisco on October 2, 2023.

The two incidents dramatically highlighted the limitations of automated vehicle technology and even advanced driving assist systems (ADAS).? That these two events occurred in the U.S., where pedestrian fatalities are inexplicably on the rise relative to the rest of the world likely contributed to the recent National Highway Traffic Safety Administration (NHTSA) mandate - FMVSS 127 - calling for pedestrian automatic emergency braking implementation by 2029.

Notably, both pedestrian crashes – the one involving Uber and the one involving Cruise – occurred at night which is a particular focal point of the NHTSA mandate.? To quote the press release announcing Federal Motor Vehicle Safety Standard 127:

“The new standard requires all cars be able to stop and avoid contact with a vehicle in front of them up to 62 miles per hour and that the systems must detect pedestrians in both daylight and darkness. In addition, the standard requires that the system apply the brakes automatically up to 90 mph when a collision with a lead vehicle is imminent, and up to 45 mph when a pedestrian is detected.”

In a presentation at the recent AutoSens 2024 event in Barcelona, thermal sensor supplier Teledyne Flir noted the safety concerns NHTSA was seeking to resolve.? The agency had previously promoted a voluntary automatic emergency braking adoption plan with the industry. The agency did not specify which sensor or combination of sensors ought to be employed to satisfy its new standard.

The Teledyne Flir presenter at AutoSens noted that NHTSA pedestrian fatality data in the U.S. shows that “the vast majority of fatalities, injuries, and property damage crashes occur at speeds above 40/km/h (25 mph), which are above those covered by the voluntary commitment.”? The NHTSA data shows that 32% (1,582) of pedestrians in the U.S. are killed annually at night by vehicles traveling between 40-50 miles per hour.

The agency has not specified the technology to be used to fulfill the FMVSS 127 requirement, but Teledyne Flir showed data derived from testing conducted by VSI Labs that revealed that a car equipped with Teledyne Flir thermal sensors fulfilled the mandatory requirements while cars equipped with monocular cameras, stereo cameras, or a combination of cameras and radar failed.

It is worth noting that crash test dummies used to complete these tests must now be equipped with thermal technology to accurately simulate real humans moving into the path of moving vehicles. The expression VRU - for vulnerable road user - is increasingly being used to identify all forms of pedestrians from walkers to bikers to construction workers and first responders.

Companies, regulators, and standards-setting bodies are all increasingly focusing on the challenge that identifying and avoiding VRUs poses to self-driving cars and advanced safety systems. The NHTSA mandate in the U.S. is perhaps the most stringent requirement yet specified by any regulatory agency.

Teledyne Flir is not the only thermal sensor supplier detecting a major market opportunity in FMVSS 127. A total of eight companies are vying to cash in including Adasky, Lynred, Owl.ai , Valeo, Magna, Seek Thermal, and Obsidian.

But Teledyne Flir is perhaps making the strongest case thanks to its work with VSI Labs. The conclusions of the study of four cars, only one of which was equipped with thermal sensors:

• Three 2024 OEM vehicles were tested to FMVSS 127 standards and all failed some night tests
• Only the thermal-enabled PAEB system passed all FMVSS 127 tests
• Test protocols using heated soft pedestrian targets are required for thermal camera testing
• Real-world corner cases are more challenging than current FMVSS protocols
• OEMs should consider utilizing thermal cameras for FMVSS 127, especially for nighttime PAEB
• Thermal cameras provide the OEM additional safety such as weather, glare, smoke, and system redundancy
• Animal detection should be considered as an additional feature with thermal cameras.

For more than a decade, thermal sensors have been lurking on the periphery of the automotive industry promising a potential safety enhancement to nighttime driving. The FMVSS 127 mandate and the demonstration of the technology's efficacy will change the conversation around meeting the challenge of collision avoidance. This is only the beginning.