{Seminar} @ CDS: 12th January: “Learning to Generate, Edit, and Arrange 3D Shapes.”

In computer vision and robotics, we often need to deal with 3D objects. For instance, we may want to generate instances of 3D chairs, edit the generated chairs using natural language instructions, or arrange them in a canonical orientation. In this talk, I will present some of our work on addressing these problems. First, I will talk about ShapeCrafter, a model for recursively generating and modifying 3D shapes using natural language descriptions. ShapeCrafter generates a 3D shape distribution that gradually evolves as more phrases are added resulting in shapes closer to text instructions. In addition, I will introduce the notions of invariance, equivariance, and ‘canonicalization’, and discuss their importance in 3D understanding. I will describe ConDor, a self-supervised method for canonicalizing the orientation of full and partial 3D shapes. Finally, I will identify future directions including opportunities for expanding 3D understanding to neural fields, articulating objects, and object collections.

Host Faculty: Prof. Venkatesh Babu