Ray Tracing for sound, the holy grail for data generation?

Ray Tracing (RT) should be a very familiar term in 3D gaming, but what might be less known is its application in rendering sound, and how it could solve the data problem in model development.

Light and sound’s transmission in air share some similar physical behavior, most importantly, what happens when it hits a surface. Therefore, it’s logical to think Ray Tracing engine could be used to do the same for sound. VRWorks from Nvidia, includes exactly that, a path tracing engine for audio. Just like RT, by calculating how sound wave changes through many different paths between the source and listener, then combine the results together we get a realistic rendering of sound at the listener location.

Before a demo, I would like to quickly share a project I did many years ago, a custom wake-word is needed to be part of the product. This is what happened:

1. 50 employee carry a recording setup and record the wake-word in various environments, office, canteen etc. [2 weeks]
2. Data cleaning [2 days]
3. Model training [1 day]
4. Model verification [1 day]

The point is, data collection is a big effort task in audio model development and within this effort, being in the required physical environment contributes a lot of the headache. This is exactly where RT for sound come in handy, by rendering realistic sound in a 3D environment.

Let's have a listen to a demo clip, using headphone is highly recommended, the purple sphere represents sound source.

Demo video clip

VRWorks engine supports customization of:

• The 3D environment (by giving it mesh file)
• Reflection, absorption & transmission coefficient of surfaces
• Sound source(s)
• Listening location(s)

With these features and assets from game development world, it’s not hard to imagine scenes like office, canteen and church can be simulated, to a very high degree. So, could this be the holy grail for acoustic data generation?