3D Mesh Generation Using Deep Learning

This is a recent project that originated as a class project for Berkeley’s CS280: Computer Vision course. The idea is to use neural networks, more precisely 3D convolutional neural network paired with GAN (generative adversarial network) to generate realistic 3D mesh-based object.

Here’s a paper on this research: ArXiv Link

car_sample_15 By Max Jiang Modelo »

Hallucinating 3D objects has been done in the recent past, with vary degrees of success using binary voxels. However I am more interested in developing a mesh-based framework that creates mesh-based objects. Mesh based objects are nice, as it is the default data structure used in computer graphics, even scientific computational applications. The trick is to use signed distance function (SDF) based voxel representation instead of binary voxels. Of course, there’s some more “magic” with detail enhancement, which can be found the the above paper.

Merrying object generators with mesh-based graphics rendering algorithms creates pure magic. Realistic yet artificial scenes:

The nice properties about arithmetics in latent space still hold. Interpolating between latent vectors give a smooth morph between shapes:

'Max' Chiyu Jiang

'Max' Chiyu Jiang

My name is Max Jiang. I'm a third year PhD student at UC Berkeley. My research interest is in Machine Learning, AI, and its applications to fluid related physics.