AIDA UHD-100A & NDI?: Immersive Video Production using Meta VR

As I submit to the clasp of the Meta Quest VR headset, a palpable sense of foreboding permeates the air, the device's rubbery face guard beginning its confining embrace, and the world begins to dissolve into abyssal darkness, then, revealing a miniature, digital landscape woven from diamonds and diodes. The fisheye lenses of the AIDA cameras intricately mold a panoramic skyscape inside the seemingly infinite void, entwining reality with phantasmagoria through the arcane art of 3D wizardry. I find myself transmuted into the world of “The Burrow”, the VR headset’s initial grasp unsettling now forcing my eyes to descend into the labyrinth of the protagonist’s psyche – an immersive fusion of the tangible and the surreal, a singular entity fusing within me, a mere spectator no more -? I metamorphose into an unwitting participant in this spectral endeavor. As the haunting cadence of Ward Meijer’s compositions reverberates into my temporal lobe, my gaze traverses the byzantine pathways of Kafka’s perception, and the narrative unfurls within the recesses of my consciousness, an enigmatic sojourn into the unknown.

This is what we thought it might have felt like if we were at “The Burrow” as Franz Kafka, but since we didn’t have tickets, we just imagined it.

The Kafka Production

In a production house somewhere in the Netherlands – Wieger Steenhuis, Silas Neumann, and Ward Meijer had the unique task of producing a performance for one of Kafka’s short stories, “The Burrow.” Neumann was in charge of creating the set, Meijer the music, and Steenhuis the digital integration. The goal was to immerse the audience in a format where they could openly explore the intricacies of the protagonist’s mind throughout the story. To achieve this, the three decided on the use of virtual reality (VR) to teleport spectators into a model world. Steenhuis researched VR technology, ultimately opting for a live filming approach using two AIDA UHD-100A cameras. The chosen cameras, equipped with 180-degree fisheye lenses, proved to be perfect for capturing a 180-degree stereoscopic (3D) image for this avant-garde project, challenging the distinctions between reality and the imaginative.

“We chose these cameras because they were actually perfect for this purpose, small, affordable, good clean image quality, and there are of lot of different lenses available.” – Steenhuis

A Brief Look into 3D Visual Immersion

Unlike traditional flat-screen productions found in movies and television, 3D filmmaking is the next frontier in cinematic storytelling. This innovative approach immerses viewers in a visually expansive experience that extends over a 180-degree field of view.

First-time users exhibit different levels of adaptation to the virtual world, with some seamlessly assimilating, while others needing an extended adjustment period. Most commonly, VR users frequently describe the experience as stepping into an alternate dimension where the lines between the physical and digital blur. Here, an illuminating visual spectacle unfolds before your eyes and around your peripherals, prompting your senses to transcend pre-existing interpretations of human perception.

This newfound awareness is due to the format of the headset displays, which provide a level of depth in a three-dimensional space that also responds to motion. Users can interact with the visual landscape by looking around in the headset, offering a sense of agency and personal engagement within the narrative.

The diverse actions of each individual in a VR experience are as vast as the multitude of possibilities for creating fantastical realms through 3D imagery. With the additional use of spatial audio and interactive controls, productions can form a cinematic experience that transcends the boundaries of traditional filmmaking. In this 3D world, artists have the unique capability to craft worlds that extend across a 360-degree plane, adding profound depth to the storytelling experience.

AIDA Impact

Integrating the UHD-100A cameras allowed Steenhuis to create an intimate performance space. Aligning both cameras together to replicate a 3D environment allowed the audience to step into the main character’s eyes. The live video feed, captured in 3D using the AIDA cameras, underwent a near real-time transformation from fisheye distortion to a stereoscopic 180-degree equirectangular format.

GPU calculations using Derivative’s TouchDesigner were employed to properly align the video feed to the headset display. The final touch involved developing software for Meta Quest VR headsets, which offered an immersive experience for the audience using the Unity Game Engine, a platform for real-time rendering and interaction. This software seamlessly displayed the video stream with a 3D stereoscopic effect in near real-time, offering a truly immersive experience to the spectators.

The combination of scale models, new media, and the AIDA UHD-100A facilitated a captivating journey into the spirit of Kafka’s narrative. The audience, equipped with VR glasses, found themselves not merely spectators on a two-dimensional perspective but as active participants in the story on a three-dimensional plane.

Results

The success of this project underscores the importance of choosing the right tools for creative endeavors. The integration of NDI? allowed for wireless transmission of the video signal from the main computer to the VR headsets, enhancing the overall experience in terms of audience immersion, production costs, and accessibility.

The AIDA cameras provided the necessary clarity, affordability, and versatility required for capturing a 180-degree stereoscopic image. The integration of NDI played a crucial role in seamlessly streaming the live video feed to VR headsets, ensuring an uninterrupted and immersive experience for the audience.

The Kafka Production showcases how the convergence of technology and art can redefine storytelling. The UHD-100A, combined with NDI integration, opened new dimensions for live productions in the performing arts, pushing the boundaries of creativity and audience engagement.

As for the wizards behind this spectacle, we tip our imaginary hats to their ingenuity.

Watch The Kafka Production Trailer

Credits:

All things analog: Silas Neumann

All things digital: Wieger Steenhuis

Audio & music: Ward Meijer

--

Check out the UHD-100A: https://bit.ly/aida-uhd-100a

To learn more about AIDA NDI products: https://bit.ly/aida-ndi

Follow us on social media: https://linktr.ee/aidaimaging

*NDI? is a registered trademark of Vizrt Group