ESA at Gamescom 2024: Exploring the Final Frontier of Gaming!

Following its hugely successful participation in 2023, ESA will be returning to Cologne, Germany, in August to Gamescom , the world’s largest computer and video games fair!

There are close links between virtual worlds in gaming and ESA’s real-world space missions. Certain technologies and techniques from the gaming industry help ESA to prepare astronauts for missions and to explore our Universe. At ESA, Virtual Reality (VR) headsets are used for simulations that immerse astronauts in settings such as the ISS or the Moon to help them get familiar with their surroundings. Another crossover from the gaming world is the use of haptic feedback , a technology through which feedback is provided through touch, used both in gaming for remote controllers and by ESA to control robot rovers and robotic arms.

In this edition, we talk with ESA colleagues who share their work with these technologies and how they are shaping our understanding of the universe, and they also give a few hints about what to expect from ESA at Gamescom.

Marius Schwinning, Internal Research Fellow at ESA's ESOC, in Darmstadt, Germany.

Marius Schwinning works at ESA’s European Space Operations Centre (ESOC ) in Darmstadt on the preparation of future lunar surface missions. This includes surface analysis, landing site selection and operational analogue experiments for projects such as Argonaut, Europe’s future lunar lander.

Can you share a particularly exciting or unexpected moment in your research journey at ESA that has significantly influenced your work?

In my first year at ESA, I had the chance to participate in the final experiment of the METERON project. We operated a lunar analogue mission at Mount Etna, an area that exhibits particularly good conditions for field trials. On-site, we had a joint team of DLR and ESA colleagues preparing the infrastructure including two lunar rovers, while I led the surface operation team in the Mission Control Room at ESOC . This three-day campaign was such an immersive experience that we all felt like we were operating an actual mission on the Moon. The knowledge gained from this and other similar experiments helps us a lot in preparing for our future lunar exploration programme.

What is a fun fact about your daily life as a Research Fellow at ESA?

I’m ESA’s technical officer for the development of a lunar crater game, thus combining two of my passions: space and gaming. What started as an innovative idea in a coffee break became an actual project. By marking craters in real lunar satellite images to extract in-game resources for a fictive lunar base, the gaming community helps ESA and the University of Applied Sciences Darmstadt build a crater database to train an AI for automatic hazard detection and around future landing sites. So, in short, gamers will help ESA to safely land on the Moon from 2031 onwards. Curious how it looks and eager to participate? Come to the ESA booth at Gamescom and try the game yourself.

If you could collaborate with any scientist or team in the world, who would it be and why?

The team of the Lunar Reconnaissance Orbiter mission. The amount and the variety of data we have received from the satellite orbiting our Moon is unprecedented and enables us today to get a very detailed understanding of the surface conditions. Without this mission we wouldn’t be able to select a landing site for our future Moon missions with the same confidence as we can do today. Directly collaborating with the team that smooths the way towards lunar exploration is an enthralling thought.

?

Noora Archer, Young Graduate Trainee in Virtual Reality Development.

As a Game Designer, UX Designer and New Media Artist and Storyteller, Noora gets really excited about space and sustainability. Her passion is designing and creating products and services that inspire awe and spur innovation. Noora is currently working at ESA, developing virtual reality solutions for future exploration missions beyond Earth.

The intersection of VR and space exploration is fascinating. Could you walk us through one of your recent projects and explain how it can play a part in future missions?

One of my favourite projects is a virtual experience called MUSE, a Modular User-Centred Spaceflight Experience. ?We utilise game design tools and platforms to create an extensively modular spaceflight experience. One of the core pillars of the work we do at my unit here in Spaceship EAC in Cologne is to inspire internal and external communication on capabilities and possibilities nested in the European space field. With MUSE, the user gets to experience what spaceflight to the Lunar Gateway space station might feel like and gets to see what the Earth and Moon look like while on board a crewed spacecraft. It is not just for inspiration, however: having a visual, reactive and interactive platform for such a complex mission concept opens vast development and research opportunities, from machine learning to systems engineering, user interface design and ergonomics studies.

You will be attending Gamescom this year, supporting the VR team at the event. What kind of experiences or demonstrations will you be showcasing?

I want to show the visitors a new addition to the MUSE experience: The Launch and Return experience, where the visitor gets to do a small mission to the low Earth orbit and return safely back to Earth, while experiencing first-hand in VR what flying a spacecraft feels like. ?

What advice would you give to aspiring game designers and UX designers who dream of working on groundbreaking projects like those at ESA?

The future of designing for space exploration may often feel distant, yet at ESA, we are currently engaged in an exciting period of exploring innovative technologies to support various space activities. Game designers and UX designers play an integral role in this technological ecosystem too, with new tools and methods from the design field. My advice to a game designer or UX designer is to seek out interesting professionals in the space sector, learn about their work, and visualise the kind of multidisciplinary project you would undertake if given the chance to collaborate with them. You may be at the beginning of something innovative!

?

Lionel Ferra, Software & Artificial Intelligence Lead.

Lionel?Ferra?heads the?Team?of Artificial Intelligence, Software, Extended Reality and Quantum Technologies in the Directorate of Human and Robotic Exploration at ESA. He was the lead instructor for the rendezvous and docking of the European spacecraft ATV and was also responsible for space robotics exams for the ISS crew. A key component of his new? team is to ensure that ESA projects become software-centric and that new technologies are used both on the ground and in space, as the digital transformation of ESA is crucial for efficiently exploring our Solar System with robotic resources and crews. As such, XR was introduced as a new training tool back in 2017 and since then numerous crews have been trained for ISS flight and on-board ISS using VR.

Can you tell us about a recent breakthrough or innovation in software or artificial intelligence that your team has developed for ESA?

Recently, our team, part of the Directorate of Human and Robotic Exploration, has developed groundbreaking XR and AI solutions for astronaut training and mission design preparation (especially for orbital Moon missions and surface operations). This innovation uses advanced in-computer vision and other technologies to support simulations of ISS missions and Moon landings, and we are starting to look at applied Mars applications. We also recently trained not only on the ground but on board ISS for more immersive and just-in-time training. The various applications we created enabled us to support the design of analogues (LUNA), prepare Lunar Gateway missions based on ISS heritage and lessons learnt, accurately recreate the lunar surface and also prepare Mars missions. These not only enhance training efficiency but also improve mission success rates by allowing astronauts, designers and ground controllers to experience and react to potential challenges in a controlled, adaptative and immersive environment.

In the context of space exploration and gaming technology, how do you see artificial intelligence playing a role in enhancing the user experience or solving complex challenges? Could you provide an example from your work at ESA?

Artificial intelligence is essential in both space exploration and gaming technology, offering unprecedented enhancements to user experiences, scientists, ground crew and flight crew as well as providing solutions to complex challenges. At ESA's XR-Lab , we've harnessed AI to develop smarter space systems and designed interactive virtual environments for training and mission design. For instance, our XR simulations for the ISS (outside and inside), Moon orbital outpost and the lunar south pole and Mars missions allow all stakeholders of a space mission to practice and refine their skills in lifelike settings, but also contribute to the public awareness of ESA space missions. This improves design, module habitability, operations and training efficiency, readiness and performance, as well as allowing us to identify and address potential issues before actual missions, ultimately increasing the mission outcomes.

Gamescom attracts a diverse audience interested in both gaming and technology. What unique aspects of ESA's software and AI initiatives will you highlight for attendees at the event?

At Gamescom, we'll highlight the innovative work of ESA's XR-Lab and focus on how our software and AI initiatives improve space mission training and exploration, especially in the area of astronaut training, space mission design and immersive environments. ?Key aspects include training systems that simulate ISS missions, south pole Moon missions, and Mars explorations with unparalleled realism. Attendees will see how these simulations provide immersive, interactive experiences that prepare astronauts for real-life scenarios that complement classical training (e.g. spacewalk or robotic arm operations). We are hoping to inspire attendees with the exciting intersection of gaming technology and space exploration, showcasing how these advancements are paving the way for humanity's future in space.

?

Nadia Lüders, ESA Branding & Partnerships Officer.

Nadia’s career started in the mid-90s, at the very beginning of the digital revolution. It was a fascinating time, witnessing the birth of the digital age. Combining a full-time job and late nights studying in communication, law and computer science set the scene for her professional life to date: using the power of storytelling with new tools as a bridge between seemingly separate worlds.

Now, Nadia leverages her unique skillset as ESA’s Partnership Coordinator. She combines her communication and licensing expertise to create partnerships across different markets including the movie and video game industries.

Imagine collaborations that bring the wonders of space exploration to life through captivating video games – that's the magic Nadia wants to create and that's why she is really pleased to attend Gamescom! She will work with multidisciplinary and multicultural teams and together explore the unknown and engage with a wide audience. Ultimately, Nadia is a bridge builder, fostering communication and creating partnerships that push the boundaries of space exploration and engagement. It's a job that's out of this world, and a perfect match for her varied experiences.

What is a story or narrative your team has developed to inspire and engage audiences with ESA’s work?

The European Space Agency's brand narrative weaves a story of exploration, collaboration and the very real benefits of reaching for the stars. We are driven by a deep curiosity about the cosmos, constantly pushing the boundaries of knowledge through groundbreaking missions. We also recognise that ESA is not alone in this journey. We believe international collaboration is a key advantage, and this is also part of our narrative: bringing nations together to achieve ambitious space goals. Ultimately, our work isn't just about scientific discovery, it's about using space exploration for the betterment of humanity, from practical applications to inspiring the next generation of scientists, explorers and dreamers.

Space is not just about improving things in the future either; it is also very much about helping people in their daily lives right here and right now. From your satellite navigation, your phone and internet connectivity to your daily weather reports, space has become ubiquitous to a point that people hardly notice just how much it already plays a role in their lives.

With space exploration and technology evolving so rapidly today, what challenges does that pose in terms of ESA's branding?

It is true that the rapid pace of space exploration and new technologies have thrown down a challenge for ESA's branding in a few specific areas. ESA needs to constantly update its messaging to reflect the latest discoveries to stay relevant and capture the public’s imagination. To amplify our reach, our branding and partnerships team collaborate with non-space related companies to reach a wider audience.

What role do fun and entertainment play in your partnership campaigns?

For me, the magic lies in how these games created by ESA’s partners generate emotions and ignite curiosity for space. A sense of awe as players race to save a space mission, the thrill of building a spacecraft and steering it through an asteroid field, or the satisfaction of creating strategies and synergies with other gamers to build a mission to Mars – exactly the same as we do in reality with other space agencies. Experiences such as these can spark a lifelong fascination with space exploration.

What are some of the key partnerships ESA has formed with game developers or tech innovators?

I will start by mentioning “Lunar Horizons ” as it is the most recently released game. It is an immersive experience set on the Moon. Released on 11 April 2024, the game was created by Epic Games, ESA and Hassell, in collaboration with Buendea and Team PWR.

You can find some of our key partnerships here .

We work with the video game industry taking into account their needs and development timelines. ESA offers a treasure trove of content and data to developers. For developers crafting realistic space simulations or fictional space adventures, ESA offers invaluable technical advice and expertise. Our knowledge ensures that the players’ experiences are grounded in real science. Beyond these direct contributions, ESA also participates in competitions that challenge gamers. This not only keeps things exciting for players, but also helps to foster a passionate community around space exploration.

What can attendees expect from ESA's presence at Gamescom 2024?

The European Space Agency expects to create more links between the space and gaming industries and to inspire the next generation of space explorers:

• Industry networking: ESA is looking to collaborate with gaming companies on future projects related to ESA’s space missions to raise awareness about ESA’s activities.
• Focus on the connection between space exploration and video games: ESA aims to showcase how its space missions are used in video games and how video game technologies are used by space industry and especially for astronaut training simulations.
• Interactive experiences: Attendees will get to try hands-on demos using game tech for space applications.
• Career opportunities: Visitors can also get information about careers at ESA.

Thomas Ormston, Spacecraft Operations Engineer.

Thoms Ormston serves as the Deputy Spacecraft Operations Manager for the Sentinel-1 mission and is stationed at the European Space Agency’s European Space Operations Centre (ESOC ) in Darmstadt, Germany. His journey into space exploration began when he joined the Mars Express Flight Control Team at ESOC, where he oversaw critical aspects such as the power, thermal and communications subsystems. Notably, Thomas spearheaded the Mars Webcam project during his tenure on Mars Express and played a pivotal role in supporting NASA's JPL Mars rovers. In 2012, Thomas started supporting ESA’s Earth Observation fleet, including piloting spacecraft such as the Earth Explorer CryoSat-2 and laying the groundwork for upcoming missions such as EarthCARE . Since 2020, Thomas has been responsible for the operational health and safety of the two flying Sentinel-1 satellites. His duties encompass navigating them through space debris hazards and ensuring they fulfil their mission objectives as integral components of the Copernicus Programme. This programme provides free and open Earth observation data to all, empowering global efforts to tackle some of the most pressing challenges facing humanity today.

Spacecraft operations involve complex systems and real-time decision-making. Have you ever faced a challenge in your work where your gaming skills were unexpectedly beneficial? Spacecraft operations are in many ways just like a computer game but with a lot, lot less action - it’s a classic “what people think I do” vs. “what I actually do” situation! In fact, it’s less about the skills you might imagine from a high action racing or flight simulator and more about the skills you might use in a strategy or adventure game. Flying spacecraft is all about balancing resources, considering what risks might appear and how best to manage them. Then of course when something does go wrong you need to learn how to prioritise but also to use the limited clues to work out what’s going on and puzzle out a solution.

How do you stay updated with gaming trends and technologies and do you find any overlap with advancements in space technology? It’s sometimes hard to draw parallels between the two: space technology can appear quite old-fashioned, because you want it to be dependable and reliable above all else – we can’t accept bugs and crashes when the spacecraft depends on it! That said, the trend of games like Minecraft and Roblox, which allow users far more control to develop and craft their own vision, mirrors the availability in the space world of ever more advanced software technologies that allow scientists and engineers to explore and implement new concepts with space technology on –the fly in a way that is more user-friendly than ever before.

What aspects of your work at ESA do you think will resonate most with Gamescom attendees? I’d actually say the aspect that would resonate most is not technological but cultural – there are so many parallels between the gaming world and our world at ESA. We’re a huge community that networks across countries and borders, united by technology and always seeking to push boundaries and overcome challenges.

Join us at Gamescom 2024!

Whether you're a gamer, developer or space enthusiast, ESA's presence at Gamescom 2024 promises to be an unforgettable experience. Come explore the cutting-edge technology and thrilling adventures that lie at the intersection of space and gaming.

ESA’s booth will be in the careers area of Koelnmesse?in Hall 10.2. Stop by and speak with our representatives about opportunities at ESA – and don’t miss your chance to try out our VR experiences! Read more about Gamescom here and we’ll see you there, gamers!

Launch your career with us! Positions are regularly being published on our recruiting site and you can set up a job alert to receive a notification when a job matching your profile is published. You can also subscribe to our careers news to stay informed on the latest news from Careers at ESA.