Deep Surface Meshes

Geometric Deep Learning has recently made striking progress with the advent of Deep Implicit Fields (SDFs). They allow for detailed modeling of watertight surfaces of arbitrary topology while not relying on a 3D Euclidean grid, resulting in a learnable 3D surface parameterization that is not limited in resolution. Unfortunately, they have not yet reached their full potential for applications that require an explicit surface representation in terms of vertices and facets because converting the SDF to such a 3D mesh representation requires a marching-cube algorithm, whose output cannot be easily differentiated with respect to the SDF parameters.  

In this talk that was presented at CVPR'20 workshop on Deep Declarative Networks, I discuss two different approaches to overcoming this limitation and implementing convolutional neural nets that output complex 3D surface meshes while remaining fully-differentiable and end-to-end trainable. I also present applications of these approaches to single view reconstruction, physically-Driven Shape Optimization, and bio-medical image segmentation.

For more details, please take a look at the following two arXiv reports https://arxiv.org/abs/1912.03681 and https://arxiv.org/abs/2006.03997