How can you use ARKit to create multi-user AR experiences?

Según mi experiencia hay 3 conceptos básicos que las redes de RA, deben de afrontar. la Interoperabilidad , la estandarización y el mapeo espacial y comprensión de la escena. Durante mi recorrido y experiencia en RA a principios del 2018 estos han sido mis principales retos. Si es cierto que espero con gran entusiasmo e ilusión La Alliance for OpenUSD (AOUSD) donde están trabajando muchas de las dificultades que nos encontramos en la RA como la interoperabilidad de contenido 3D y La estandarización del ecosistema 3D que permita a los desarrolladores y creadores de contenido describir, componer y simular proyectos 3D

In our experience, the key to successful multi-user AR experiences with ARKit lies in precise synchronization of AR data, including device orientation and virtual object interactions. Local networking, using Wi-Fi or Bluetooth, is effective for real-time interactions in small-scale environments, where ARKit excels in minimizing latency. For larger scale projects, remote networking over the internet ensures consistent user experiences across distances. This understanding of ARKit's networking capabilities is crucial in crafting immersive AR environments.

Hi Everyone We just released an AR gaming platform offering this feature called ARIDDLE If you are interested in testing it with me we can talk. We solve all the key points you mention in the different contributions. Players can be anywhere. Regards

The best way to create a successful client focused multiuser AR experiences in ARKit is to ask three questions at the start of the project. Where is the audience based? What tech are they using? What is the clients creative needs to ensure that their message is easily understood and engaged with? ARKit is inherently very good at handling the complexities of multiuser applications. The limitations are all external factors such as network speed and reliability and user experience. Minimizing user download time and data sharing by creating content that works within network limitations whilst preserving the clients message and intent is key. This ensures that magical first touch win for users that frames their perception of excellence.

In my work, I’ve found that the choice of network type greatly depends on the desired AR experience. Local networks often have lower latency, which is crucial for real-time interactions. Remote networks provide more flexibility and scalability, beneficial for large-scale, geographically distributed experiences. The challenge lies in seamlessly integrating AR content into both network types to ensure a consistent and immersive user experience.

In our ARKit projects, we've effectively used its multi-user features for collaborative AR environments. The ARWorldMap data structure is key, allowing users to align in a shared 3D space. This enables consistent virtual object placement across devices. The ARParticipantAnchor further enhances interaction by showing other users' positions in the AR scene, promoting collaboration. ARCollaborationData keeps devices in sync, updating their ARWorldMap and ARParticipantAnchor. These features are vital for engaging, interactive multi-user AR experiences.

ARKit turned my solo AR project into a shared journey. ARWorldMap let our devices sync in a virtual space, like a collaborative dance where virtual objects moved in unison. The ARParticipantAnchor added a personal touch. Suddenly, my friend's device became part of my augmented reality, making it feel like we were in the same space, even though miles apart. ARCollaborationData played the role of a silent coordinator, ensuring our AR experiences evolved harmoniously. It was more than tech; it was like co-authoring a digital story, where our interactions shaped a shared narrative. ARKit transformed the digital into a collective adventure, making technology a bridge between our worlds.

Los principales retos son el comportamiento de los modelos 3d con la comprensión del entorno, mapeo, anclajes y las físicas en los modelos 3d creadas con colaiders. También Problemas con las superficies reflectantes, poco contraste, a mayor meshing menos rendimiento, Near clipping plane. Principalmente los mayores problemas que nos encontramos con RA, fue proyectar imágenes virtuales en ambientes muy luminosos, dado que no funcionaban bien. Si es cierto que esto va avanzado muy rápido y xreal o varjio ya lo tiene bastante avanzado respecto a magic leap o ARkit. Aunque creo que firmemente que esto quedara resuelto con la entrada de Vision pro y AR Kit dará un salto enorme.

ARKit comes with various tracking method, WorldTracking helps us to track and create a AR world as soon as we start the ARSession, having a proper place with lots of unique tracing points helps in accuracy of ARKit. Each anchor point helps uniquely identify a spot in ARKit. ARParticipantAnchor is a special type of that anchor which tracks each users position and we can sync data between those users using ARCollaborationData. WorldTracking , ARParticipantAnchor and ARCollaborationData all together brings about a really engaging multi-user ARexperience.

In ARKit, the ARWorldMap feature enables devices to share and align in a common 3D space, enhancing multi-user interactions. ARParticipantAnchor plays a key role, allowing users to see and interact with each other’s devices in real time. ARCollaborationData further facilitates session synchronization across devices, ensuring consistent and updated shared experiences. These elements combine to create seamless, engaging multi-user AR experiences.

In creating multi-user AR experiences with ARKit, we've faced challenges like network latency, reliability, and security. Latency issues often lead to AR scene mismatches, disrupting user experience. I've observed that even slight delays can cause significant confusion. Network reliability is another hurdle; unstable connections can lead to data loss or abrupt disconnections, breaking the immersive AR experience. Security is paramount too. In one project, we had to ensure that AR data, including sensitive user interactions, was encrypted and secure to prevent data breaches or identity compromises. Addressing these challenges is crucial for a seamless and secure multi-user AR experience.

In a recent project, network latency briefly disrupted the harmony, introducing glitches that momentarily disoriented users. Unreliable network connections acted like an unexpected pause in the performance, disrupting the flow of the AR narrative. Security became personal as I safeguarded the privacy and integrity of the AR experience. It's not just a technical concern; it's like being a guardian, protecting the essence of the narrative we were creating. These challenges aren't just technical hurdles; they're the unexpected twists that make the journey of crafting multi-user AR experiences both complex and captivating

Creating multi-user AR experiences poses challenges like network latency, leading to scene inconsistencies and user frustration. Reliable networks are essential to maintain experience continuity, with failures risking data loss. Network security is critical; vulnerabilities can expose sensitive data or invite malicious attacks. For example, compromised AR data could threaten user identity and scene integrity. Addressing these issues is vital for a seamless, secure AR experience, necessitating robust solutions to ensure user engagement and safety. This complex task requires careful balancing of technical proficiency and user-centric design.

One thing we've found helpful in our ARKit projects is implementing specific network solutions to enhance AR experiences. For latency issues, optimizing network performance through data compression and encoding has proven effective, ensuring a smoother AR experience. To tackle network instability, we've incorporated error correction and retransmission strategies, enhancing the resilience of our AR applications. Additionally, prioritizing network encryption using advanced cryptographic methods has been crucial in protecting AR data against security threats. These strategies collectively have been instrumental in delivering robust and secure multi-user AR experiences.

In my experience, AR shared experience has certain limitations, no of users you can share with, and the distance if you’re sharing AR Data across devices locally. And surely sharing AR experiences across networks would add latency issues making certain AR Frames almost irrelevant and performance issues which just adds to the overhead. However sharing across networks allows it to be platform independent.

We've integrated native tools like the MultipeerConnectivity framework - a powerful tool provided by Apple that enables iOS devices to discover and communicate with nearby devices. For cross-platform experiences, we've used solutions like Photon, bridging the gap between iOS and other platforms. Additionally, we employ network monitoring tools like Wireshark to optimize performance in varying network conditions, compensating for ARKit's limited network quality assurance.

Como he indicado en uno de los artículos anteriormente antes, creo que La Alliance for OpenUSD (AOUSD) con Apple, el grupo Khronos con Nvidia Pixar,Adobe y Autodesk la tecnología de Arkit tendrá un gran peso y la colaboración entre estándares 3d como uszd o gltf creo que darán esta compatibilidad entre plataformas y dispositivos

With all respect to ARKit, if you use their built in multi-user feature you are limiting yourself to the Apple ecosystem. I always develop for cross platform using other socket based technologies that provide more than just scene synchronization, for example voip and avatar agency. Beyond our in-house solution I would recommend, normcore, agora or photon each has their own advantages and disadvantages.

For my customers in the automotive industry, ARKit can offer revolutionary AR-based navigation and safety features, like windshield HUDs for real-time updates. It can also transform maintenance, allowing mechanics to visualize internal components and diagnostics with AR overlays, simplifying repairs. In aerospace, ARKit aids in designing and maintaining aircraft, with engineers using AR for 3D part modeling and technicians for guided repairs. Pilots can benefit from AR-enhanced navigation and system checks. These applications in automotive and aerospace highlight ARKit’s potential to blend digital information with real-world scenarios, enhancing efficiency and safety.

In my experience the Shared Session with ARKit has many churn points. Which makes the solution hard to adopt. To counter this problem we need to focus on networking over the internet instead local networks. The “join session” should be as fast as possible, easy and well guided as users can understand and follow through. The applications for this feature are endless and with further adoption of MR devices we will see an increase in multiplayer applications and games for AR.