Computer Vision: What KPIs for Performances Evaluation?

The definition of good image quality for an #ADAScamera, entirely dedicated to computer vision and embedded in a vehicle,?is still a subject of?debate. It seems obvious that the key performance indicators for camera evaluation need to be redefined in light of the new score of #computervision. Since?2022, #DXOMARK has launched an ambitious research program?to evaluate various image quality?metrics in regard to algorithms' performances.?

The aim of the research program, in collaboration with a French university, is to correlate the results of three metrics -- CTA, CSNR, together known as CPI (Contrast Performances Indicator), and FRC?-- with the detection rate of a license plate-reading algorithm based on a #neuralnetwork.

• Contrast Transfer Accuracy (CTA):?quantifies the ability of a camera to reproduce a contrast with fidelity,
• Contrast Signal to Noise Ratio (CSNR):?quantifies the ability of a camera to distinguish objects,
• Frequency of Correct Resolution (FRC):?characterizes the resolution limit of a camera, with respect to a detection threshold.

Some of the preliminary results are indicated below.