WWDC 2023 – Intro to Spatial Computing
Here we are, learning more about spatial computing! You can check the video below:
Also, here's the link of the original publication in my personal blog: https://brunocamps.com/2023/06/07/wwdc-2023-intro-to-spatial-computing/
Windows, Volumes, and Spaces
Let’s walk through a few concepts.
By default, spatial apps launch in the Shared Space. It’s like the desktop on your Mac where you launch and interact with multiple apps at the same time.
Windows. Each app has one or more windows. They are SwiftUI scenes, resizable and can be reflowed. They can contain traditional views and controls and you can match 2D and 3D content.
Volumes. They allow an app to display 3D content in defined bounds. We can reposition volumes in space and view from different angles. They are SwiftUI scenes and use RealityKit to display 3D content.
It’s funny to see Apple’s engineers talk about Windows during the keynotes. I can see some funny looks on their faces.
Once you’re familiar with SwiftUI, it’s fairly easy to work with spatial elements like the code below:
Talking about Spaces: there’s a difference between Shared Space and Full Space. A Full Space allow you to make your app’s windows, volumes and 3D objects are the only ones appearing across the view.
You can use passthrough to ground content?in the real world and keep people connected?with their surroundings. This makes you feel that virtual objects really belong in the room you’re in.
Fully immersive space allows your app to fill up the entire field of the view.
Gestures
Gestures are well integrated with SwiftUI, like taps, long presses and drags, so people can interact directly to your 3D scene elements (interactions with RealityKit entities).
领英推荐
ARKit Skeletal Hand Tracking: this allows you to use gestures, app-specific hand inputs to your apps.
Collaboration
SharePlay: when people using VisionPro do an activity together.
SharedContext: participants in a SharePlay session experience content in the same way through SharedContext. There are SpatialPersona templates to customize how people experience the content.
Privacy
The Privacy session has to do with permissions the app may request. For example, hands data (skeletal hand tracking).
How to develop spatial apps
In theory, developers won’t need to have a VisionPro device right away to develop a spatial app. The xrOS device simulator will be integrated to Xcode and we’ll be able to interact in a virtual space from our Macs.
In the screenshot above, we can have a glimpse of the structure of a spatial app. In the RealityViews group, the developer defines Earth, Moon, Sun, Satellite, Telescope and Starfield as separate views and they will interact with each other in the main view. For those familiar in working with SwiftUI, this project pattern is not new.
Reality Composer Pro
This is a brand new tool. Preview and prepare 3D content for your spatial apps. You’ll be able to visualize your objects and how they fit together in the scene.
Particles is a feature added to RealityKit, and you can use the Composer Pro to author and preview them. You can add particles to your scenes to add movement and other possibilities.
At this point, we start to see the new concepts of spatial computing that are part of visionPro’s SDK. Apple’s augmented reality development kit – ARKit – didn’t feature these new concepts.
In the session Explore Materials in Reality Composer Pro, we’ll learn more about all of it. Also Meet Reality Composer Pro (which I’ll probably post later on, check this article).
Spatial apps: how to get started
You can either develop a brand new app in the spatial context or bring an existing app to the spatial world.
iPhone and iPad apps are fully supported, but I have a feeling that iPad apps will look way better.
Here are a few code snippets from the presentation:
Model3D
Gestures
We see a mix of existing SwiftUI gestures and spatial only gestures:
RealityView: A new view that can be added to your scene. A SwiftUI View with Entities, also integrated with RealityKit.
For this session, this is it! Once we can get our hands on the SDK, I’ll be able to post more about it! Don’t forget to check this article as the main reference for the WWDC 2023 series.