Determining the Repeatability of Automotive Audio Testing pt. 2

Introduction?

Previously we investigated the repeatability of Head and Torso Systems (HATS) in an automotive environment. These systems are often used across many use cases spanning both telephony and audio quality to give objective measurements on the subjective question of “how does this sound?”.??

This seemingly simply question can get quite complex as companies strive to make high quality products with limited time and resources. There are many tools that are available to aid in the process. Ranging from sound level meters and real-time analyzers with a single microphone, to full acoustic measurement systems that will output insightful results such as phase, delay, frequency response, distortion, and mean opinion scores.?

How can we ensure that as we dive into the toolbox that confidence is ensured in the measurements taken, but also that it is representative of what would perform well on the market? The standard measurement system in the automotive industry is to set-up an array of microphones (typically a 6-microphone array as stated by Earl Geddes in “Localized Sound Power Method” Engineering Report from 1980). The microphone data points are then averaged to give spatial resolution of varying user sizes and ear locations. This gives insight to all relevant data that is needed to tune a car to a certain target, but it may not give the whole picture of user perception. This is where HATS can bridge the gap.?

A point of concern could be what deviation can be expected between a 6-microphone array when compared to HATS???

Methodology?

To accurately simulate a scenario that one might use for final validation or a tuning process in a vehicle, an HMS II.3 LN HEC (HATS), 6-microphone array (consisting of 6 calibrated ?” Free-Field Microphones), a HEAD acoustics labCORE with a Bluetooth module (coreBT2), and HEAD acoustics ACQUA was used.??

The HMS was mounted on a seat mount and placed in the driver’s seat. For each measurement group, the HMS was completely removed from the vehicle, then replaced. Measurements were taken at 3 different seat positions matching the location of the 6-microphone array. They are noted as the 5th, 50th, and 95th percentile seat positions. They are approximately the ear location of humans that match the respective size percentile.?

Once replaced, the overall level was checked, and adjusted, ensuring that it was 85dB[SPL] (+/- 1dB) and two measurement runs were recorded.??

Similarly, a 6-microphone array was calibrated and mounted in the driver’s position. For each measurement group, the array was completely removed from the vehicle, then replaced. The seat position was not changed for the array.??

?The microphone array is labeled 1-6. Their positions corresponding to the HMS recording is indicated below:?

1. Top Left (Outboard) = HMS Left Ear @ 95th percentile seat position?
2. Top Right (Inboard) = HMS Right Ear @ 95th percentile seat position?
3. Middle Left (Outboard) = HMS Left Ear @ 50th percentile seat position?
4. Middle Right (Inboard) = HMS Right Ear @ 50th percentile seat position?
5. Bottom Left (Outboard) = HMS Left Ear @ 5th percentile seat position?
6. Bottom Right (Inboard) = HMS Right Ear @ 5th percentile seat position?

Once replaced, the overall level was checked, and adjusted, ensuring that it was 85dB[SPL] (+/- 1dB) and two measurement runs were recorded. The same test signal was used for both the array and the HMS.??

The source file chosen for this study was the MDAQS sequence (the Multi-Dimensional Audio Quality Score), seen below. It consists of two sine sweeps (left, then right), followed by excerpts of different genres of music. For our purposes this is both a dynamic and applicable signal when discussing Automotive Audio.?

From each measurement run, the Active Speech Level (ASL), Frequency Response, Loudness and MDAQS score were calculated when applicable. The testing was performed on a mid-level crossover. The source signals were played via Bluetooth and recorded via the measurement system under test (HMS or Array).?

Results?

Standard Seat Position vs. Microphones 1 & 2?

The starting point and gold standard for tuning any acoustic system is frequency response. The following graph shows the averaged frequency response of all 20 runs for Microphones 1 and 2 of the Array.?

Below shows the averaged frequency response of all 20 runs for Left and Right Ear of the HMS.?

Both measurement systems were calibrated to the same overall level and show similar characteristics across the spectrum. The graphs below show the corresponding Array Channel to the Channel of the HMS.

Theres is some differences in Frequency Response between the HMS and the Array. Characteristics of the environment are present in both the results, but more appreciable in the HMS (i.e. flatter response from the right ear indicating more diffuse field; left ear shows fluctuation from possible reflections and proximity to loudspeakers).?

The following plot shows a difference between the HMS and the Array from 20Hz to 20kHz.?

?When calculating the difference between the two plots, it is important to note that there are fundamental differences between the two measurement systems and more of an introspective approach to what is the end goal of the data collected. Both devices give valuable and confident results to design and validate an audio system. However, the utilization of HATS allows for the inclusion of variables that are not available with an array such as the Head-Related Transfer Function (HRTF). This is also present as we investigate the other microphone locations. The same analysis will be shown.?

Middle Seat Position vs. Microphones 3 & 4?

The following graph shows the averaged frequency response of all 20 runs for Microphones 3 and 4 of the Array.?

Below shows the averaged frequency response of all 20 runs for Left and Right Ear of the HMS.?

There is less deviation between the Array and the HMS that can be accounted for from environmental factors. The middle seat position and Microphones 3+4 are in a more diffuse environment when compared to the standard seat position and the lower seat position. The graph below shows the delta between the HMS in the middle seat position and Microphones 3+4 of the array.?

The variation between runs is tighter for the HMS compared to the standard seat position and comparable from the Array between locations.?

?

Bottom Seat Position vs. Microphones 5 & 6?

The following graph shows the averaged frequency response of all 20 runs for Microphones 5 and 6 of the Array.?

Below shows the averaged frequency response of all 20 runs for Left and Right Ear of the HMS.

There is a similar deviation between the Array and the HMS when comparing to the middle seat position. The graph below shows the delta between the HMS in the bottom seat position and Microphones 5+6 of the array.?

There is higher variation run to run in both the HMS and the Array at the lowest position. But overall, the HMS and Array show comparable results.?

?

MDAQS?Results

One of the benefits of utilizing binaural recordings via HATS is opening the availability of post-analysis metrics. At times, basic measurements (e.g. frequency response) may not represent the system in its entirety and are unable to predict human perception. To combat this, Jury tests could be done with a listening panel, doing comparisons on recordings and evaluating them subjectively. This is a reliable approach but does not meet the criteria for a quick and cost-effective solution to assess audio playback quality. An available metric is MDAQS – the Multi-Dimensional Audio Quality Score. The HEAD acoustics developed algorithm calculates a Mean Opinion Score (MOS) ranging from 1 (worst value = bad) to 5 (best value = excellent).?

Using the MDAQS source signal and recording binaurally, the MDAQS algorithm calculates four MOS values for the DUT:??

• MOS-T for Timbre?

• MOS-D for Distortion?

• MOS-I for Immersiveness??

• MOS-O; A combined MOS for overall quality??

Using the 20 runs we recorded and analyzed from each seat position with the HMS, the average MDAQS score was calculated.?

Conclusion?

Audio tuning and validation is an integral step in product design and customer satisfaction. The modern system designer and engineer has many tools available at their disposal, but they all come with benefits and drawbacks.?

A multiple microphone array gives a spatial average of the pertinent acoustic zones for many customers. It’s a reliable and repeatable way to take good amounts of data simultaneously and gives broad characteristics of the quality and validity of the system. However, there is more upkeep and set-up time to ensure that all microphones are calibrated and positioned properly from event to event. It also does not account for the seatback or headrest in its measurements. This is crucial as technology is advancing to accommodate Zoning concepts within a vehicle cabin such as headrest speakers, which an array cannot accurately capture the effect and response of.?

Using a HATS gives objective results of how a customer would experience the environment and allows for faster set-up time and repeatable results. It also allows for psychoacoustic analysis and other sound/audio quality metrics to be analyzed such as MDAQS.??

These can then be implemented for ‘quick checks’ from vehicle to vehicle and ensure that a certain product line meets the standard for the manufacturer, as well as the customer. The drawback is in order replicate the spatial averaging that an array allows for is to physically move the seat to separate positions which could increase time depending on what is desired.?

Test methodologies should strive to be as sophisticated as the technology being tested. The debate on which system should be used is reduced to what governing philosophy is present for tuning. Modern data acquisition front ends are capable of handling both the array as well as HATS. There should be an awareness for the pitfalls and ‘traps’ that are present in either method. However, there should be no concern whether the results that are being recorded are accurate, but rather what is needed to be captured and subsequently evaluated.

References:?

Geddes, Earl; Blind, Henry; The Localized Sound Power Method [PDF]; Ford Motor Company, Dearborn, MI; Paper 2127; 1984 Available: https://aes2.org/publications/elibrary-page/?id=11627?