Less Polygons, More Particles?

I went to the The Metropolitan Museum of Art last weekend and took the above 3D scan with my phone in 30 seconds from the Luma AI app. The speed and resolution of these captures are insane. Go check out the Gaussian Splat (Gsplat) scan here .

I've also taken other fun scans! Here is my small NYC apartment.

I definitely need to go read up more, as I don't claim to be an expert on all things 3D Capture NeRF, Gsplat, etc. Especially these days as the technology is moving so fast. Its tough to keep up with Photogrammetry , NeRF , SMERF , BLiRF , and Gsplat , among others. My colleague Dan Cotting mentioned he needs to remind people he is not Dr. Seuss when talking about these, and I think many of us in the industry share the same sentiment. However, it seems tough to argue that the potential impact of this technology on computer graphics and humanity will be anything but substantial.

If you are reading this and agree or disagree, I'd love to hear from you!

I have been interested in 3D reconstruction for several years now, and can say with confidence it has never been this easy and fast to capture, process, and distribute a 3D scan of an object or space with such high visual fidelity. These Gsplat "particle" based (loosely defined) assets are an amazing innovation.

As I mentioned in the Digitization to Digital Twin blog post for Amazon Web Services (AWS) , I believe these techniques can be different tools for different jobs, recognizing that there can be substantial overlap in use-cases between them.

Realization

Sometimes a 3D mesh/polygon based output is necessary, however I have realized, (as have many others in the industry) that all use-cases do not (and will not in the future) require mesh based 3D assets. As long as you can utilize the 3D asset within the desired interactive environments downstream (web/real-time engine), and provide the graphic representation necessary for the use-case at hand, you should be all set. In fact, in some cases it's looking like there can be substantial optimization and graphic fidelity benefits to not using mesh/polygon based assets.

You can read a bit further on 3D mesh/polygon model optimization here .

Tech Maturity and Integrations

As mentioned, this stuff is moving quickly. The tools in the ecosystem around these technologies are maturing quickly, especially editing tools and integrations. There are now integrations to bring Gsplats into web viewers such as threejs , and Luma even has their own WebGL Library . There are also paths to pull these splats into real-time engines such and Unity and Unreal, which means we will most likely see more of these assets being leveraged in AR/VR, E-commerce, Digital Twins, and other types of experiences.

It's going to be fun to continue to watch this technology mature over time. I'm looking forward to reading up further and playing with these tools. What an exciting time to be a 3D nerd! I will most likely post more of these articles about different topics I am interested in. Feel free to follow along, and thanks for reading! :)

#VR #AR #gsplat #spatialcomputing #aws #amazonwebservices #amazon #innovation #pacePrototyping #prototypingWithAWS