A Brief Discussion on the Requirements of LED Displays for Virtual Production

In today’s high-tech film and television production industry, virtual production is gradually becoming a mainstream technology. Especially the use of LED screens for virtual shooting has become an indispensable part of film production. This article provides a brief overview of virtual production technology and its main application types, explaining the role of LED displays in LED virtual production projects. It also analyzes how LED displays impact the effectiveness of virtual shooting and explores the future development of LED display systems in virtual production. The improvement of overall display quality in LED screens will further drive the advancement of LED virtual shooting technology.

1????? Introduction

In recent years, LED virtual shooting technology has gradually emerged in the film and television industry and has been widely adopted. This technology combines LED backdrop walls with computer graphics technology, providing producers with greater creative flexibility while addressing issues such as blue or green screen spillover commonly seen in traditional virtual shooting. Consequently, with the development of the LED display industry and the rapid iteration of modern information technology, more LED virtual shooting projects have come into being, achieving a “what you see is what you get” shooting effect, as illustrated in Figure 1.

In February 2024, the China Optics and Optoelectronics Manufactures Association officially released the world’s first standard for LED displays used in Virtual Production (VP), titled “Specification for LED Display Systems Used in Virtual Production (VP)”1. This standard fills a gap in the domestic virtual production industry by establishing a grading system for the display optical performance of LED screens used in virtual production and standardizing the use of LED displays in virtual shooting. Therefore, further exploration of the specific parameters that LED displays must meet to align with the true needs of the virtual production market holds significant reference value for the development of LED display products aimed at applications in the virtual production field.

2????? Virtual production

2.1???? Application Types of Virtual Production

The most commonly used virtual production methods today primarily involve the use of high-performance LED displays as backdrop walls. Through real-time rendering engines, multi-screen synchronization, and real-time rendering techniques, three-dimensional scenes of high visual quality are rendered onto the LED backdrop. These scenes are adjusted in real-time via rendering engines, synchronizing with on-set lighting, mechanical devices, and other filming tools. The camera directly captures the real actors’ performances and props, which are seamlessly integrated with the LED backdrop in real-time. This innovative film-making approach achieves a “what you see is what you get” result.

However, depending on the specific application scenarios of virtual production, it can be divided into VP (Virtual Production) studios and xR (Extended Reality), as illustrated in Figure 2. The main technical difference between the two is that xR involves extended reality, whereas VP does not. Their differences in definition, implementation methods, and application forms are outlined in Table 1. In the market, most references to LED virtual production generally refer to virtual production in a broad sense, without specifically distinguishing between these two types. However, VP tends to have higher demands on LED displays compared to xR. Therefore, this article primarily focuses on analyzing the requirements for LED displays in virtual production by examining their role in LED virtual shooting.

Table 1 ?Differences Between VP and xR

2.2???? LED Virtual Production Display System

Before discussing the role of LED displays, it is essential to first clarify the composition of the entire LED virtual production system. Typically, an LED virtual production system consists of an LED display system, a camera tracking system, a camera and its supporting system, a media server system, and a lighting system. Additionally, the topological relationships between these systems must be understood. Figure 3 illustrates the principle of LED virtual production, highlighting the subsystems within this system and their functions. For example, the LED display system must receive environmental images output by the media server and display these images in coordination with the actors’ performances. These performances are captured by the camera as image information, which is then combined with the virtual environment by the media server to produce the final output.

Therefore, based on the analysis of previous LED virtual production cases, the role of LED displays in virtual production can be categorized into three main types:

1. As a Background: The LED screen replaces the traditional green screen, serving as the visual carrier of the virtual world during virtual shooting.
2. As a Light Source: The LED screen substitutes for traditional lighting, providing illumination during virtual production to simulate natural light.
3. As an Interactive Display Medium: The LED screen serves as a medium for actors to interact with during filming.

Given the different positions and uses of LED screens in virtual production, the requirements for the displays also vary. For instance, in a virtual studio, the background screen, as seen in the first and third scenarios, needs to be captured by the camera again, which imposes certain requirements on the overall display quality, as illustrated in Figure 4. The background image needs to be as smooth and realistic as possible. In contrast, the second scenario often involves the use of a ceiling screen in virtual production, as shown in Figure 5, where higher brightness is required to meet the lighting needs under different environments.

Considering that the display performance of LED screens directly impacts virtual production, and that many factors influence this performance, particularly due to the presence of cameras, certain display parameters are subject to more stringent requirements. Therefore, this article focuses on the display performance requirements of background screens that are directly captured during virtual production. For instance, when conducting virtual production, the images on the LED screen are filmed by the camera to serve as part of the virtual scene, necessitating that the display screen delivers outstanding visual quality. But what specific display dimensions can be used to evaluate this performance?

3????? Multi-dimensional Analysis of Display Performance Requirements

（1）Pixel Pitch

The pixel pitch of an LED display refers to the physical distance between two adjacent pixels, which are the smallest light-emitting units on the screen. A smaller pixel pitch results in a finer and more detailed image display. This also means that the minimum allowable shooting distance for the camera can be closer when filming, providing greater flexibility in the camera’s movement during virtual production. As a result, the quality and versatility of the captured footage are enhanced. In the current market, the mainstream LED displays used for virtual production typically have a pixel pitch of P3 or smaller, with P2.5, P1.9, and P1.8 being common choices. There is even a trend toward developing products with an even smaller pixel pitch.

（2）Brightness

Brightness is one of the most straightforward parameters of an LED display and has a significant impact on the quality of the captured image in virtual production. This is because the choice and setting of the camera aperture during virtual production must take into account the display’s brightness. Only when the screen brightness is sufficiently high can the camera operate with a wider aperture range.

First, it’s important to consider the position and role of the screen in virtual production. For example, in a virtual studio, the background screen typically operates indoors, where the required brightness is generally between 1200 and 1500 nits.

For the ceiling screen in virtual production, a higher brightness is necessary to create different lighting effects for various indoor scenes. For instance, Absen’s PR series, with models like PR3.9 and PR5.2, can achieve a brightness of up to 6000 nits, making them more suitable for use as ceiling screens.

（3）Color Gamut

The color gamut refers to the range of colors that an LED display can reproduce. This range is determined by multiple factors, including the characteristics of the LED diodes, the capabilities of the accompanying video processors, brightness, and other related display parameters. In the realm of virtual production, the commonly adopted standard for evaluating color gamut aligns with that of the digital cinema industry, namely DCI-P3. The extent of this gamut reflects the color reproduction capabilities of LED displays.

DCI-P3, short for Digital Cinema Initiatives P3, is a cinema-grade color gamut standard widely used in digital cinemas. It prioritizes human visual experience, aiming to match as closely as possible all the color requirements presented in films. Notably, it covers 25% more area than the sRGB color gamut standard, as illustrated in Figure 6. When employing virtual production techniques for film creation, creators prefer that even when the LED displays are captured by cameras, they can accurately present any desired color. This necessitates that the DCI-P3 color gamut coverage be as extensive as possible. Currently, LED screens used in virtual production typically achieve over 90% DCI-P3 coverage.2

CRI (Color Rendering Index)

In addition to color gamut, another standard for evaluating color reproduction capabilities is the CRI, or Color Rendering Index. CRI measures the ability of a light source to reveal the colors of objects in a way that is faithful to their appearance under natural sunlight. It is a comparative metric that assesses how accurately a light source can reproduce the colors of objects when compared to a standard light source, such as daylight. While color gamut focuses on the range of colors, CRI reflects the quality of color reproduction.

CRI, sometimes referred to as CIE Ra, is measured using the general color rendering index Ra, with values ranging from 1 to 100. A higher CRI, indicated by a larger Ra value, signifies better color rendering by the light source, meaning it more accurately reproduces the colors of objects. This also means that when filming with a camera, the colors of the subjects will appear more true-to-life, making them look more realistic.

As a result, CRI is an essential factor for lighting designers in virtual production, helping them decide the appropriate lighting effects to match the scene requirements. To achieve better shooting results in LED virtual production, the display screens must have a high CRI performance, with higher Ra values being preferable. Currently, the industry’s average Ra level is around 55, indicating that there is still significant room for improvement in the future.

（5）Frame Rate

Conventional LED displays typically operate at a frame rate of 60Hz to 120Hz. However, the film and media industry often requires specialized shooting techniques, such as frame doubling, frame interpolation, and various frame rates, making a high frame rate essential for LED displays used in virtual production.

This need arises because it’s uncommon for animated videos to natively achieve frame rates as high as 240fps. Therefore, in practical projects, frame doubling and interpolation are used to render four 60fps video segments simultaneously, allowing the display to refresh these images at over 240Hz. This process is often synchronized with the camera’s shutter adjustment function, enabling multi-camera, multi-angle shooting of the same object. It allows for capturing different depth-of-field images of the same object against various video backgrounds, thereby enhancing efficiency and reducing production costs in virtual film-making.

A higher frame rate means smaller animation increments, resulting in smoother visuals with no noticeable ghosting or screen tearing. This smoothness is especially apparent when playing slow-motion video animations. As illustrated in Figure 7, the images captured at different display frame rates show that the frame at 251Hz is noticeably smoother. Therefore, achieving a frame rate of 240Hz or higher is crucial for ensuring the smooth visual presentation in virtual production.

(6) Viewing Angle

Typically, an LED display offers optimal visual performance within its specified viewing angle. However, once the viewing angle exceeds this range, color shift, also known as off-axis color shift, can occur due to visual changes. This is a natural issue for displays, and while it cannot be completely eliminated, its impact can be minimized.

In virtual production, if an LED display can provide a wide viewing angle of ±80° (160° total) in both horizontal and vertical directions, it can maintain good color consistency and brightness even when viewed from extreme angles. This capability allows cinematographers to move more freely, capturing the desired footage without being restricted by the shooting environment.

The off-axis color shift caused by viewing angle can be evaluated by observing how the color temperature changes across different angles. For example, by aiming a photometer at the center of a single display panel, rotating the panel from -80° to +80°, and measuring the color temperature variation at the center point, one can assess the consistency. Taking Absen’s PR2.5 product as an example, the test results show that the color temperature at the center point fluctuates slightly around the standard value, as illustrated in Figure 8. This demonstrates that when viewing the display from the left or right, no significant color shift is noticeable, allowing the camera to freely capture images from various angles within the 160° wide viewing range.

(7) Contrast

To simulate the realistic light and dark contrasts of the real world, an LED display needs to have a high contrast ratio. This enables the screen to present a richer range of color tones and more nuanced light and shadow details, making virtual scenes appear more three-dimensional and lifelike. This quality allows color to truly become part of the filmmaker’s visual language.

High contrast can be achieved by effectively reducing light reflection. For instance, selecting high-brightness, ultra-low-reflection LED components, such as four-in-one LED diodes, or using ultra-black masks with light-trap technology, can help achieve higher contrast. These features minimize the impact of screen surface reflections on lighting setups, reducing the workload for lighting designers. Additionally, they can effectively mitigate the effects of moiré patterns. As shown in Figure 9, optimizing and reducing moiré patterns can significantly enhance the on-camera visual quality during LED virtual production, providing excellent visual effects through the camera lens.

(8)Bit Depth

The bit depth of an LED display significantly impacts the completeness and quality of the image it can display. Higher bit depth allows the display to handle more information, resulting in smoother color transitions and reducing the likelihood of noticeable color banding. In virtual production, it’s crucial that the display’s bit depth processing capability matches the high bit depth of the input image to ensure that the image is rendered accurately and in full detail.

（9）Overall Image Quality Rendering Capability

Beyond the eight key performance parameters previously discussed that affect the visual quality of LED displays during filming, there are four additional critical factors: refresh rate, scan rate, grayscale, and HDR.

Refresh Rate: A refresh rate of 7680Hz can effectively prevent the appearance of scan lines in the image. Scan Rate: A lower scan rate, such as below 8-scan, allows the display to refresh a complete image faster, which is more camera-friendly and results in smoother images with fewer artifacts. Grayscale: A grayscale level of 16-bit ensures smoother color transitions and minimizes issues like color banding, thereby accurately reproducing colors. HDR (High Dynamic Range): Supporting HDR allows the display to render more realistic and detailed images, enhancing the overall visual experience.

As shown in Figure 10, these factors are crucial in virtual production environments, where the LED background screen is also a subject being captured by the camera. The elector-optical performance of the display must be optimized to ensure that the final captured image is of the highest quality, making the demand for comprehensive image quality rendering capabilities of LED displays increasingly stringent.

4. Future for LED Virtual Production

Analyzing the various factors influencing virtual production effectiveness reveals that LED virtual production technology has significant potential for further upgrades and innovations, particularly in areas such as color accuracy and contrast. Enhancing these aspects can lead to even more realistic and vibrant visual effects.

With the advancements in artificial intelligence (AI) and computer graphics technology, LED virtual production is expected to become increasingly intelligent and automated. This will streamline processes such as scene construction, lighting adjustments, and post-production. Consequently, the accompanying LED display systems will need to evolve to match these advancements, ensuring that virtual production continues to deliver exceptional results and attracts more film and television projects.

Additionally, virtual production's capability for real-time rendering is advantageous for creative flexibility and modifications, free from the constraints of natural conditions. This reduces the time and manpower costs associated with venue changes and set modifications, providing a more efficient and realistic filming method for the industry. As these technologies and techniques continue to develop, LED virtual production will likely become an even more integral tool in the filmmaking process.

5. Conclusion

LED virtual production technology is rapidly evolving, with broad market prospects and substantial development potential. As technology continues to advance and applications expand, LED virtual production is increasingly becoming a crucial technique in the film and television production industry. By examining the factors that impact the display effects of LED screens from the perspective of virtual production creators, and considering the current performance and future development potential of display screens, filmmakers can fully appreciate the value of these screens. Furthermore, ongoing improvements in LED display technology will meet more complex and diverse filming needs, enhancing the versatility and effectiveness of virtual production in the future.

?

Reference

[1] T/COEMA 18S—2024. LED Display System Specifications for Virtual Production (VP) [S]. China Optics and Optoelectronics Manufacturers Association, 2024.

[2] Liu Zhiyi. Analysis and Research on Core Technical Indicators of LED Movie Virtual Production Systems [J]. Modern Film Technology, 2023(7):26-32.

?