What Types of KPIs are being validated for ADAS?

At Ottometric, we are committed to advancing the future of automotive safety and efficiency through innovative validation solutions for Advanced Driver Assistance Systems (ADAS). Our platform is designed to streamline the validation process by leveraging cutting-edge capabilities such as data integrity checks, synthetic KPI-truth generation, and advanced anomaly visualization. By integrating these tools, we help manufacturers not only meet but exceed the highest standards of safety and performance, delivering reliable and robust ADAS features to market faster.

In this article, we explore the four critical categories of Key Performance Indicators (KPIs) that are essential for ADAS validation: System, Feature, Function, and Sensor. Understanding the distinct roles these KPIs play is crucial for a comprehensive approach to testing, ensuring that ADAS technologies are rigorously validated to enhance safety and performance on the road.

We will highlight the difference between them and how an effective validation process for ADAS must consider all types of testing.

Manufacturers can get the best of both worlds by separating these categories, resulting in a more comprehensive and rigorous validation. This leads to more reliable and superior ADAS features that help in making the driving experience safer and more efficient. These KPIs at various levels are important for development and validation of robust ADAS that would improve safety as well as the driver experience using the vehicle.

System KPIs

System KPIs are essentially macro-level yardsticks that gauge the overall efficiency and efficacy of the complete ADAS entity. These indicators would typically offer a wide-angle view of how well the system comes together and operates cohesively as a singular unit,? often taking into account various factors like reliability, user delight, and even holistic safety considerations. The broad picture painted by System KPIs is more than a mere sum of its parts; it’s an indicator of the symphonic harmony (or lack thereof) playing out within the intricate depths of an ADAS system.

Examples:

1. System Reliability: Measures the uptime and failure rates of the ADAS.

o ? Example: The ADAS should have a mean time between failures (MTBF) of at least 10,000 hours.

2. User Acceptance: Evaluates user satisfaction and acceptance of the ADAS.

o ? Example: A user satisfaction survey may reveal that 95% of users find the ADAS intuitive and reliable.

3. Overall Safety Impact: Assesses the reduction in accident rates due to the implementation of ADAS.

o ? Example: Vehicles equipped with ADAS should show a 30% reduction in rear-end collisions compared to vehicles without ADAS.

Feature KPIs

Feature KPIs are unique to each ADAS component (adaptive cruise control, lane departure warning, automatic emergency braking) and address whether each feature is doing what it is supposed to do well or not.

Examples:

1. Adaptive Cruise Control (ACC) Accuracy: Measures the precision with which ACC maintains a set distance from the vehicle ahead.

o ? Example: ACC should maintain a following distance within ±0.5 meters of the set distance at all times.

2. Lane Departure Warning (LDW) Response Time: Evaluates how quickly the LDW system alerts the driver when the vehicle unintentionally drifts out of its lane.

o ? Example: LDW should alert the driver within 0.5 seconds of lane departure detection.

3. Automatic Emergency Braking (AEB) Activation Time: Measures the time it takes for AEB to activate once a collision threat is detected.

o ? Example: AEB should engage the brakes within 0.3 seconds of detecting an imminent collision.

Function KPIs

Function KPIs are specific numerical measurements that help assess the functioning quality of each particular feature within the Advanced Driver Assistance System. Typically, they pertain to how well individual sensors or components accomplish their task in support of the feature.

Examples:

1. Camera Sensor Front Camera Module for Lane Detection: Measures the clarity and detail captured by the camera sensor used for lane detection.

o ? Example: The camera sensor should have a field of view of at least 100° to accurately detect lane markings.

2. Radar Detection: Assesses the scope of the objects which the radar sensor can detect.

o ? Example: The radar sensor should reliably detect vehicles, stationary objects and vulnerable road users.

3. Lidar Detection: Evaluates the precision with which the lidar identifies objects.

o ? Example: The lidar sensor should detect vehicles, stationary objects and vulnerable road users.

Sensor KPIs

Sensor KPIs are like the performance report cards of sensors playing the role of data sources in an ADAS system; these sensors can be considered the 'sense organs' (eyes and ears) of a vehicle. The reason why the functionality of ADAS features is largely determined by how well these individual sensors perform.

Examples:

1. Accuracy: Pinpoint precision in measurements taken by the sensor regarding distance, position, or other pertinent data.

o ? Example: The radar sensor should detect distances with an accuracy of ±0.1 meters.

2. Resolution: For cameras and LiDARs, how much detail can the sensor capture?

o ? Example: The camera sensor should have a resolution of at least 1080p for clear object detection.

3. Range: How far can the sensor reliably detect objects?

o ? Example: The lidar sensor should detect objects up to 200 meters away.

4. Reliability: How consistently does the sensor function under different conditions?

o ? Example: The sensor should maintain 99% reliability under varying weather conditions.

Summary of Differences:

• System KPIs: Provide a holistic view of the ADAS performance, focusing on overall reliability, user satisfaction, and safety improvements.
• Feature KPIs: Target the performance of specific ADAS features, assessing how well each feature accomplishes its intended purpose.
• Function KPIs: Focus on the detailed performance of individual components or functions that enable ADAS features, often related to sensor performance.
• Sensor KPIs: Delve into the performance details of individual sensors, measuring aspects such as accuracy, resolution, range, and reliability.