Michael Niemeyer

I’m Michael, a PhD student in the field of computer vision / machine learning in the AVG group at the Max Planck Insitute for Intelligent Systems. Currently, I am a research intern at Google Brain.

My research focuses on 3D vision. I am interested in how machines can infer 3D representations from sparse observations. Further, I am big enthusiast of neural scene representations. I currently investigate how scenes are best represented for machines using deep neural networks.

I studied BSc Mathematics at the University of Cologne, Germany. During my Bachelor studies, I spent one year at the University of Barcelona, Spain, funded by the ERASMUS program. Following my interest in computer science, I then moved to Scotland to gain my Master’s degree in Advanced Computer Science at the University of St Andrews. In October 2018, I started my PhD in computer vision / machine learning in the AVG group at the Max Planck Institute for Intelligent Systems in Tübingen, Germany, under the supervision of Andreas Geiger.

Feel free to reach out to me via e-mail for any inquiries!

Mail Twitter Scholar Github LinkedIn Work

news

Jun 21, 2021	We recieved the CVPR’21 best paper award for our GIRAFFE project
May 12, 2021	I was invited to give a talk about Generative Neural Scene Representations at Amazon Research
Apr 29, 2021	We released an explanatory video, code, etc. for our CVPR’21 project GIRAFFE: http://bit.ly/giraffe-project
Mar 31, 2021	We released a new preprint CAMPARI: Camera-Aware Decomposed Generative Neural Radiance Fields
Feb 28, 2021	Our GIRAFFE paper is accepted to CVPR’21 as an oral

selected publications

Please see the publications tab or Google Scholar for a complete list.

Occupancy Networks: Learning 3D Reconstruction in Function Space

Mescheder, Lars, Oechsle, Michael, Niemeyer, Michael, Nowozin, Sebastian, and Geiger, Andreas

In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) 2019

Abs PDF Supp Blog Code Poster Video

With the advent of deep neural networks, learning-based approaches for 3D reconstruction have gained popularity. However, unlike for images, in 3D there is no canonical representation which is both computationally and memory efficient yet allows for representing high-resolution geometry of arbitrary topology. Many of the state-of-the-art learning-based 3D reconstruction approaches can hence only represent very coarse 3D geometry or are limited to a restricted domain. In this paper, we propose Occupancy Networks, a new representation for learning-based 3D reconstruction methods. Occupancy networks implicitly represent the 3D surface as the continuous decision boundary of a deep neural network classifier. In contrast to existing approaches, our representation encodes a description of the 3D output at infinite resolution without excessive memory footprint. We validate that our representation can efficiently encode 3D structure and can be inferred from various kinds of input. Our experiments demonstrate competitive results, both qualitatively and quantitatively, for the challenging tasks of 3D reconstruction from single images, noisy point clouds and coarse discrete voxel grids. We believe that occupancy networks will become a useful tool in a wide variety of learning-based 3D tasks.
Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision

Niemeyer, Michael, Mescheder, Lars, Oechsle, Michael, and Geiger, Andreas

In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) 2020

Abs PDF Supp Blog Code Poster Video

Learning-based 3D reconstruction methods have shown impressive results. However, most methods require 3D supervision which is often hard to obtain for real-world datasets. Recently, several works have proposed differentiable rendering techniques to train reconstruction models from RGB images. Unfortunately, these approaches are currently restricted to voxel- and mesh-based representations, suffering from discretization or low resolution. In this work, we propose a differentiable rendering formulation for implicit shape and texture representations. Implicit representations have recently gained popularity as they represent shape and texture continuously. Our key insight is that depth gradients can be derived analytically using the concept of implicit differentiation. This allows us to learn implicit shape and texture representations directly from RGB images. We experimentally show that our single-view reconstructions rival those learned with full 3D supervision. Moreover, we find that our method can be used for multi-view 3D reconstruction, directly resulting in watertight meshes.
GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields

Niemeyer, Michael, and Geiger, Andreas

In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) 2021

Abs Project Page PDF Supp Code Poster Video

Deep generative models allow for photorealistic image synthesis at high resolutions. But for many applications, this is not enough: content creation also needs to be controllable. While several recent works investigate how to disentangle underlying factors of variation in the data, most of them operate in 2D and hence ignore that our world is three-dimensional. Further, only few works consider the compositional nature of scenes. Our key hypothesis is that incorporating a compositional 3D scene representation into the generative model leads to more controllable image synthesis. Representing scenes as compositional generative neural feature fields allows us to disentangle one or multiple objects from the background as well as individual objects’ shapes and appearances while learning from unstructured and unposed image collections without any additional supervision. Combining this scene representation with a neural rendering pipeline yields a fast and realistic image synthesis model. As evidenced by our experiments, our model is able to disentangle individual objects and allows for translating and rotating them in the scene as well as changing the camera pose.