Immersive audio in large-scale sports production

Nuno Duarte

Olympic Broadcasting Services (OBS), Spain

ABSTRACT

Olympic Broadcasting Services (OBS) was created by the International Olympic Committee (IOC) in 2001 in order to serve as the Host Broadcast organisation for all Olympic Games, Olympic Winter Games and Youth Olympic Games. As Host Broadcaster, OBS is responsible for delivering the pictures and sounds of the Olympic Games to billions of viewers around the world.

For the Olympic Games Tokyo 2020 and Olympic Winter Games Beijing 2022, OBS produced for the first time the sounds across all of the sports and the Opening and Closing Ceremonies in 5.1.4 immersive audio. This new native standard audio format was adopted as part of OBS’s transition to Ultra High Definition (UHD) High Dynamic Range (HDR).

?????????????????? OBS produced the live coverage of two back-to-back Olympic Games less than six months apart

?????????????????? OBS overcame numerous challenges caused by the COVID-19 global pandemic, including operations and crowd restrictions

?????????????????? Overall, OBS produced immersive audio for the live coverage of more than 100 sports events, relying on a team of 300 audio engineers and the support from hundreds of broadcast engineers and producers

?????????????????? Additionally, OBS produced pre-Games content, including title sequences with music

Introduction

The broadcast and film industries are arguably the central arena for immersive content creation and distribution.

The early adoption of immersive audio or tri-dimensional sound in cinema and TV series has been a game-changer, helping usher broadcasters into a new era for sound. Over the past decade, the broadcast industry has been exploring the new opportunities offered by Next-Generation Audio (NGA). The emergence of a few ground-breaking immersive audio technologies and listening experiences (Dolby Atmos, MPEG-H, DTS:X, VR/AR, AI), together with the continuous development of multi-channel sound techniques and roll-out of new audio codecs, has contributed to reshaping the landscape of this field, accelerating the mass market adoption of immersive audio with plenty of home sound devices now available that are capable of reproducing immersive audio.

In live sports production, immersive audio first debuted in football leagues before undergoing further experimentation in the coverage of other sports events. With its long track record in pioneering broadcast innovations and aiming at delivering high-quality content, OBS decided to adopt 5.1.4 immersive audio as its new audio standard with its transition to UHD HDR production for the host broadcast of the Tokyo 2020 Olympic Games in 2021.

Initial considerations

The Olympic Winter Games Sochi 2014 marked the first time OBS experimented with immersive audio as part of the live coverage. At Ice Hockey, OBS installed and ran a parallel audio control room, where an immersive mix base was produced from the host 5.1 audio mix, audio splits and an immersive microphone, then encoded with NGA software for a binaural listening. The outcome of this first experiment was extremely positive and helped to define the roadmap for including immersive audio in large-scale sports live productions. Back in 2014, several immersive audio techniques, technologies, encoders, decoders and playback devices for monitoring and end users were available. OBS compared and tested all of them in order to determine how these existing solutions could meet OBS’s requirements.??

Because of its scale and complexity, the Olympic Games is unlike any other live sports production environment. Imagine planning the audio coverage for more than 30 sports/disciplines, across more than 50 fields of play, taking into account the specifics for each sport/discipline and each venue – it is a tremendous effort.

To transition to immersive audio, OBS had to define a new sound format, concept and design. There were many considerations involved including both audio and non-audio factors, in addition to elements specific to an Olympic Games operation.

Audio factors

Firstly, OBS had to consider audio factors:

?????????????????? Equipment to be used (different audio consoles, speakers, meters)

?????????????????? Specific requirements from Rights Holding Broadcasters (RHBs) using different audio encoders: Atmos, MPEG-H and ACC

?????????????????? Implementation period, as the installation had to take place over a maximum of two to three days

?????????????????? Quality control process that could be performed at different points of the contribution and distribution chain

Non-audio factors

Secondly, OBS had to consider other factors that had a direct impact on the audio coverage:

?????????????????? The specific nature of the event itself, the Olympic Games being a unique multi-sports broadcast where different sports events take place, simultaneously, over a period of two weeks

?????????????????? The specific requirements for every sport/discipline as each has different characteristics (i.e. indoor, outdoor, “in box”, “in line”, “in land”, in the water etc.) and requires a specifically designed sound coverage

?????????????????? The different types of venues (i.e. small or large venue, stadiums, temporary venue, Golf course, swimming pool, with or without roof, with or without catwalk, etc.)

Other considerations

Finally, there were other considerations linked to the particularities of the Olympic Games’ operation, including:

?????????????????? Working with various production teams coming from different countries and with different experiences

?????????????????? Collaborating with different International Federations (IFs) and taking into consideration each sport’s set of regulations

?????????????????? Collaborating with the Organising Committees regarding venue management and venue access

?????????????????? Handling specific requests from RHBs

?????????????????? Assessing the impact on the overall broadcast infrastructure

?????????????????? Keeping the costs related to producing the audio signals in the new format (additional equipment and facilities) within the budget

?????????????????? Assessing the impact of the lack of spectators inside the venues

Research and development phase

Considering all of the aforementioned points, OBS led a research and development plan over the course of six years, which had a threefold focus:

?????????????????? Ambient coverage with three-dimensional microphones

?????????????????? Mix and distribution of the sound “ blocks” of the sound field

?????????????????? Monitoring (consoles, speakers and meters)

Two important events/milestones were critical in defining and implementing immersive audio for the Olympic Games:

?????????????????? Olympic Games Rio 2016, Opening and Closing Ceremonies: OBS delivered 5.1 plus splits of .4 which were encoded by United States broadcaster NBC in Atmos and distributed to their premium end users

?????????????????? Winter Youth Olympic Games Lausanne 2020: OBS provided an audio mix in 5.1.4, which was produced remotely and then encoded in MPEG-H, with interactive audio available to the public through the Olympic Channel app and website

Immersive audio at the Olympic Games: Concept and Format

Concept

After in-depth research and testing, OBS developed the following immersive audio concept:

?????????????????? Based on the sound design and capture, not using any Digital Signal Processor (DSP) or processing devices, but only microphones including two new microphones specifically developed for the Olympic production environment (referred to as 8.0 and 4.0)

?????????????????? Using a 5.1.4 channel base, without additional or special DSP or devices to recreate objects and mixes in the channel base such as beds for distribution encoding

?????????????????? Using Pulse-Code Modulation (PCM) audio, relying on a discrete audio signal flow that allows for routing and verification at? any point within the broadcast system path without any special/unknown encoder/decoder/interface.

?????????????????? Ensuring a fast implementation by the production teams, resulting in short training for primary audio engineers (A1s), no additional audio crew and short and/or no extra rehearsal production time. Also, within the COVID-19 pandemic situation, it was important to avoid hands-on training sessions.

?????????????????? Facilitating easy delivery to the RHBs, with an agnostic format that can be used by any broadcast organisation, letting them use their own codecs and according to their specifications

?????????????????? Ensuring future safe content, enabling the use of all the content as per the current distribution model, while archiving the codec without compromising the sonic quality when used through a different model in the future

Format

OBS selected 5.1.4 immersive audio as its new audio standard, compliant with encoders used by the Olympic Games’ RHBs and easy to accommodate within OBS’s existing broadcast infrastructure. It meant adding four height audio channels, providing high-level sound field, and thus the third dimension. There was no major impact when it came to the audio signal path/mapping.

Figure 1 – Audio mapping

? Figure 2 – Audio blocks

OBS also produced a broadcast stereo mix, using the same sound design sub-mix audio blocks, while maintaining the same sonic quality and allowing the sound mixer to be focused on each block rather than a significant number of different mixes.

New equipment and infrastructure

With the new concept and format defined, it was important to keep the impact on the OBS broadcast infrastructure and operations at a minimum. However, four main changes were required:

New microphones?

OBS developed two specific microphones that could independently capture the high layer to cover the third dimension for the sound field.

To minimise the impact on the OBS operation, these new microphones were to have the following characteristics:

?????????????????? They shouldn’t have any power active parts

?????????????????? They shouldn’t have DSP and/or require any specific adjustments

?????????????????? They should be easy to replace, either partially or in their entirety

?????????????????? They should use the same mounting gear as regular microphones

?????????????????? They should be limited to 8 or 4 capsules

?????????????????? The capsules should have already been used in regular production/microphones.

OBS worked closely with its provider to develop and test these new microphones (8.0 and 4.0) to achieve the desired level of performance.

Competition venues were divided into two types: Type A venues used for outdoor sports and Type B venues referring to indoor venues. The 8.0 microphone was deployed to Type A venues while, for Type B venues, OBS could rig a stereo space pair and hang a 4.0 microphone at a high level.

?

?

Figure 3 – Sound field per venue type

?

These new microphones simplify all the rig operation and the guarantees operationality during all the the events,during the long periods.

As per the mic array design the four (4) different quadrants in both layer cover the entire sound field and on the mix are assign for the different quadrants/ channels of the 5.1.4.

?

Figure 4 – An immersive 8.0 microphone was specifically designed by OBS to capture the sounds of the Olympic Games.

Production Audio Control Room Set-up – Speakers, Meters and Audio Console

Most of the OB vans or light production units (flight packs) already had their own speaker set-up in the audio control rooms, equipped for stereo and 5.1 production. It was necessary to upgrade all of units to allow for an immersive mix. A new cluster of speakers for the upper layer had to be installed to monitor both the stereo production and 5.1.4 mix, following the recommendations from a study on speaker monitoring in real production unit situations[1]. The main recommendation was to align the upper speakers with the 5.1 lower layer as much as possible in order to minimise any adjustments with delays and the issue of different Sound Pressure Levels (SPLs) from each channel/speaker.

Figure 5 – 5.1.4 set-up (left) vs. standard production audio control room set-up (right)

A different speaker placement was expected due to the varying physical spaces of each audio control room in the production units. In the end, more than 40 audio control rooms installed the upper layer, most of them as permanent installation for future productions.

Figure 6 – Audio control room set-up inside different production units during the Olympic Games

?

Figure 7 – Loudness levels

?

?

In such a large production environment, there are different audio mixers from various generations. For each of them, different monitoring path/operational solutions had to be designed. However, in all cases, the A1s had to be able to monitor stereo 5.1 and immersive audio while having all other monitoring capabilities required for sports live production.

Audio meters for immersive audio is a critical tool due to the many differences in production and the first instrument to merge sound across all live production. The meter should be able to measure and analyse each layer independently and between them, at the same time, and be visible to the audio engineer (A1) and Audio Production Quality Control (APQC) supervisors.

OBS recommended the level for each layer for all the sports which could be monitored by the different meters. The A1s and the APQC supervisors made adjustments during the event when required.

?

?

?

Figure 8 – Audio meter

Audio Production Quality Control (APQC) Set-up

An Audio Production Quality Control (APQC) room and two audio technical rooms were installed at the International Broadcast Centre (IBC), fully equipped for stereo and 5.1.4 immersive audio monitoring and troubleshooting.

Figure 9 – Audio Production Quality Control (APQC) set-upduring the Olympic Games Tokyo 2020

The speakers set -up? follow? a study? done in cooperation with Fraunhofer institute and OBS speakers provider, who consider the ITU – R BS 2051—3 and BS775

The speakers location also had consideration the venue sound field quadrants and layers according the microphones used and sound design for the different sports .

The speakers distance from the supervisor also considered a SPL and the meters loudness reference.

ITU – R BS 2051—3 -E -? Table 7

?

?

?

?

?

?

STUDIO RECOMMENDATIONS FOR 3D-AUDIO PRODUCTIONS – FRAUNHOFER and GENELEC – Technical Paper

The technical rooms focused on monitoring and guaranteeing the technical quality and OBS specifications of all of the audio channels from all venues, both for the main and back-up feeds and for the stereo and immersive mixes. They also ensured the audio signal quality throughout the contribution and distribution chain.

The APQC was where the sound supervisors, continuously in touch with all the A1s, monitored the mixes from an artistic point of view and merged all of the productions for a consistent sound “image” across all the different sports. The room was located adjacent to the OBS Production Quality Control room, allowing the sound supervisors to communicate at any given time with the OBS Chief Content Officer and Coordinating Producers regarding sound design adjustments and/or issues to be solved in order to achieve the results required.

The support provided by the APQC supervisors to the A1s, as well as their ability to react quickly and solve problems, were critical and fundamental to delivering a high-performance and successful production.

Four discrete audio channels

Four discrete audio channels were added to the contribution and distribution audio mapping and signal flow. They had a minor impact on OBS’s broadcast infrastructure as they did not require a special audio channel encoder. They could be monitored from the production unit or Technical Operations Centre (TOC) / IBC equipment at any point of the chain. The stereo and 5.1.4 mixes, from the moment they were generated within the production unit until they were delivered to the RHBs, were never altered.

Figure 10 – Audio signal flow during the Olympic Games

CONCLUSION

The adoption of 5.1.4 immersive audio as part of the OBS Host Broadcast production was a success. The full implementation ran smoothly and without a significant impact on the overall broadcast project. It proved to be a step-forward innovation which offered RHBs a new and unique product, with a high degree of consistency across the entire live production.

With its low overall impact on broadcast infrastructure, the roll-out of immersive audio technologies in large-scale sports production is particularly sustainable. Also, it is a “future-safe” product as broadcast organisations will be able to easily adapt whatever is produced now in this standard to new platforms and with different encoding specifications as technological advances continue to be made.

There is still a learning curve for all involved in mastering the new technology. In Tokyo and Beijing, OBS provided broadcast engineers with valuable training and hands-on experience of immersive audio across different areas of the broadcast operation – those legacy skills will be extremely useful not only in future Olympic Games but also to help the industry in general.

Building upon this solid concept and format, OBS is now working towards developing new listening experiences in order to render even more details from each Olympic sport. The focus has now shifted to the creative production.

?

[1] Fraunhofer/Genelec/NDU, Studio Recommendation for 3D-audio production, 2007