How do you use 3D models in your AR app?

Another model type not metnioned here is USDZ, with the launch and focus of iOS devices such as the new iPhone 15 and Apple Vision Pro it is important to keep in mind what file type will work natively or within an application (web or app based).

I would suggest to use 3D format which has been recommended by framework or tool we are using. Let’s say if we are using apple’s framework and Xcode tool, they do recommend using usdz format for 3D objects.

Selecting the right 3D model format for your AR app involves a trade-off between present practicality and future innovation. While OBJ, FBX, and GLTF are tried-and-true options, the ever-evolving world of AR suggests that GLB and USDZ are leading the way. These formats are well-suited for cutting-edge AR and WebAR experiences, offering the efficiency and realism that modern AR demands. On the other hand, FBX, a more established format, offers compatibility across a wide range of platforms and tools. It's a dependable choice for those who prioritize current industry norms and ease of use

If you're an emerging designer in XR, learning the benefits of various 3D file formats is an absolute must, especially if you're using multiple different tools that may have differing requirements. For example, when working in an industrial company, your organization may have a huge library of STL files from CAD programs which aren't natively compatible with most AR platforms. For this, you may need to determine a model conversion process to get the right format before you can start prototyping.

When creating an AR experience the 3D tool needs to be able output the formats that you want to use in your experience. For most AR experiences, all of the major 3D creation tools can be used to create and optimize models. If using a web-based or social platform your models that you create will need to be relatively low poly, depending on the target device/s. The file size is very important in ensuring that you meet the platforms target requirements and have fast load times.

When diving into the world of 3D models for your AR app, the choice of a 3D tool is critical. Blender is a free and open-source gem with a supportive community, while Maya, although more challenging to learn, shines in professional settings. On the other hand, web-based 3D modeling platforms offer a user-friendly, web-based option. Your decision should align with your goals: Blender for a supportive community, Maya for professional aspirations, and web-based 3D modeling platforms for a quick and easy start.

In addition to common tools like Blender (free, open source) there are some emerging, low-cost tools like Plasticity (free trial, $150 one-time cost) which offer their own benefits and are worth trying out!

In some cases like serious games domain, instead of look, functionality might be more important. If your requirements are focusing more on cleanness of your application, you can also strip all textures and use single color 3d models that will increase performance widely since the application will use less gpu memory and fragment shader overhead will be reduced.

One of the key things I have found is to ensure you are empathetic to the user when looking at model size and deployment. If you are deploying a heavy, complex, and animated 3D model via a web based experience most if not all users will have to be on wifi and still wait a fair amount of time. Meanwhile with AR Applications running on phones or more powerful head mounted displays these same models will be rendered seamlessly and quickly.

When it comes to optimizing your 3D models for your AR app, think of it as streamlining your app's performance and user experience. It's like decluttering your digital space while making sure everything runs smoothly. Reducing polygon count, shrinking texture sizes, and simplifying animations are your tools in this digital spring cleaning. Whether you're using Blender's Decimate modifier or Maya's Reduce tool, or even getting hands-on by manually tidying up your models, the goal is to strike the right balance between quality and efficiency. Your users will thank you for quicker loading times and smoother interactions.

Social AR platforms (think Meta Spark AR, Snap Lens Studio, or TikTok Effect House) are widely popular, but because they are intended to be run almost instantly on a device, the file sizes (and therefore, download time) are usually restricted. There are tools available from these AR platforms that can make this optimization easier, such as Meta's Blender plug-in for Spark AR. Depending on the type of 3D model you're using the size can differ greatly. If it's a 3D scan, the model clean-up process can be more involved depending on the quality of your 3D scan, and so methods may be different, as well. One hack I use sometimes is to reduce the texture size (or completely replace it) on massive models I can't further reduce polygons!

When venturing into a new framework and platform, the first stop for all of us is immersing ourselves in the documentation. Think of it as our foundational guide for the journey ahead. When we hit a snag or two, we turn to the wealth of knowledge found in community forums, where fellow enthusiasts provide valuable insights and solutions. And if we ever feel the need for a little extra guidance, we can simply watch experienced experts in action through readily available resources like videos. It's the way we navigate the intricate path of learning and mastering new technology.

One thing to keep in mind is the level of experience you want for your user. Some of the ideas above about building an experience may lack functional or full compute power of a device. Looking at application based experiences that are build in UnReal or Unity may help drive great user experience and leave a longer lasting impression.