Digital twins, 3D simulations and the collaborations that drive us

Timmy Ghiurau is at the forefront of the tech revolution at Volvo Cars.

He works within the Open Innovation Arena (OIA) with a group of people that think in decades rather than model years, specialising in unconventional and cutting-edge technology. Unsullied by the term ‘impossible’, any fresh idea has space to grow into a business opportunity of the future.

Their office environment embodies the work done there – exploratory, norm-breaking, and slightly mysterious to outsiders. Classic Scandinavian interiors are traded for exposed steel scaffolding forged to hoist a Broadway show lighting rig and office plant life swapped with false walls composed only of the digitally blank slate of bright-green fabric and vining cords to power virtual reality adventures.

To dig deeper into what goes on in this branch of the business, we sat down with Timmy for a chat about the work he does in the OIA.

Hi Timmy, thanks for joining today. Can you tell us a little about what you do at Volvo Cars?

I work as an innovation leader focusing on emerging technologies such as real time 3D, virtual twins, extended reality, artificial intelligence, as well as exploring future scenarios for the company.

As a creative technologist who has been active in the worlds of music, fashion, start-ups, media production and photography. I like to collaborate internally and externally to provoke and push the boundaries of technologies and democratise tools to build the future together.

Wow! You’ve been involved in a diverse combination of industries. What’s your background and what led you to working with us?

I am from Transylvania, Romania, and moved to Copenhagen to study music originally, but artist life was hard, so I went to study engineering, innovation, and entrepreneurship at Aalborg University in Copenhagen. There I met the Unity founders that shared my dreams to democratise digital tools that enable everyone to create what they want.

After working with various start-ups, and game industry companies, I moved to the Volvo Cars office in Copenhagen as a master’s thesis student in 2014. There I learned about cognition, user experience, and how to use my skills and knowledge to build safer cars.

Though I wouldn’t call myself a car person, I quickly fell in love with the automotive industry.

Interesting. So, from being a non-car person to falling in love, what about this industry sparked your interest?

I think the great spark for innovation lies at the intersection between creative thinking and critical thinking with influences coming from art, science, engineering and design. At Volvo Cars, I can stay true to my dream of democratising tools, leading the initiatives around virtual experiences, simulation, and collaborations we have with various institutes and partners in the field.

My focus these days is on futurist thinking, innovation management, building, leading and investing in teams that bring together creative and engineering forces.

On the track of what you’re doing now, you work in the Open Innovation Arena (OIA) at Volvo Cars. Can you explain what the OIA is and what your team does for the company?

Two years ago, we were asked to think about Innovation at Volvo Cars, and how to organise around it. As a result, we formed our team in the Research and Development department but serve the whole company.

We think of the OIA as a school of thought – similar to an art school. We have a set of principles and strategies, and we prepare new competences, simulate futures and structure collective imagination and intelligence.

Our mission is to help Volvo Cars prepare for many possible futures. We connect the fragmented dots of ideas, knowledge and insights outside and inside the company, into actionable shapes. Sitting at the intersection of foresight, data science, design and software development, the OIA focuses on emergent knowledge building through exploration and experimentation.

We try to spread an innovation culture across the company and explore breakthrough ideas that generate positive outcomes for individuals, for society and for the planet.

Aside from helping various departments utilise emerging technologies, we also explore possible new business models, and scout for start-ups that we can collaborate with and possibly invest in.

Digging a bit deeper into some of your work, Volvo Cars has used 3D modelling for years, and your team works closely with them. Can you describe these 3D models and why we use them?

Through the partnership we have with Unity, we built an internal simulator that is a complete virtual vehicle. In this digital world, we can add context to the car by simulating scenarios to improve our safety features, integrate lidar sensors and train our cameras. This is a good way to test the scenarios that are too rare or too dangerous to test in real life.

In the test environment, we can compare and analyse how virtual models, or digital twins, perform on our test track and compare how those virtual models are performing in relation to our physical tests on the real-life track. We can then fuse them to deliver best results.

We can apply the uses of these digital twins in our user experience research – enabling various organisations and stakeholders to collaborate, explore, and co-develop our products and services.

While we’ve used these models internally, we recently opened many of them to anyone outside of the company. What’s the reasoning behind that?

We can immerse users from all around the world to experience the product with help of 3D simulator, that is fully interactable.

Our collaborations with partners such as Unity, Nvidia, Varjo and other tech companies have sparked a demand for us to share some of our APIs, 3D models and datasets with other parties. Doing this would enable faster development of tech in our cars and also teach communities how to integrate with our technologies and build on top of our services. We created our Developer Portal (a tool kit with resources and documentation to support developer communities) where anyone at universities, research institutes, start-ups and tech interested individuals can find what they need in one place.

Together with Unity, we released a template of a showroom with our XC40 Recharge, highlighting Unity’s HD rendering capabilities. This template enabled millions of Unity developers to have an out-of-the-box, ready to use car model so they could develop car configurators, AR and VR experiences, or games within Unity.

With growing interest in the 3D models that we create, we launched a test track template that is now available through the Unity hub. We put the Volvo XC40 Recharge, on a digital twin of our test track H?llered, where based on a point cloud scan, and with the help of the creative team form unity, we were able to have an accurate representation of our track.

What’s the benefit for people outside of Volvo Cars interacting with the templates made by us?

As there is more and more tech and software in the car, not everyone has access to the actual product when they develop or simulate features. By connecting to the virtual twin though, they are able to emulate those same features in the tool.

The test track template highlights the new terrain and road tools provided by Unity, where users now can develop a car game, test and model various sensors. This can help with our autonomous vehicle and safety features while also testing the vehicle dynamics and user interaction on the virtual test track.

It is the first time when someone has openly released both an accurate 3D car model and representation of test facilities.

With the contents of our templates, now students, start-ups, hackathons, developers, designers develop new features, new experiences, powered by a game engine and real time 3D technology.

It is important to include the users, the developers, and potential customers in our research and development. We encourage the community to not only share what they develop, but also engage in potential collaborations. We also want to use the tool to interact more within the developer communities.

You and your team work with some exciting things! What is something you do that people wouldn’t think that a car company works with? And do you have any cool tech that you get to play around with?

People often think, ‘’How can push boundaries of these technologies?’’ By opting to use game engines and extended reality, we stand out in our field. And by collaborating with artists, museums, film makers and public institutions to get better insights, generate new narratives and collaboratively define the future we want.

Within the OIA we work with and develop a lot of interesting and novel technologies. Some exciting technologies are GPT-3*, which can be used to create virtual assistants and smarter AI, and diffusion models which can generate new type of designs and art. Another favourite of the team is mixed reality – where in collaboration with Varjo and Unity, we were able to develop a platform that could drive a real car on a test track and digitally overlay the design of another car model or driving environment in real time. We are trying to push the boundaries of this platform which, who knows, maybe even leads us to developing a time machine.

*GPT-3 – Generative Pre-trained Transformer 3 is a language prediction model that uses machine learning to produce human-like text.