[1906.01568] Photo-Geometric Autoencoding to Learn 3D Objects from Unlabelled Images

We show that generative models can be used to capture visual geometry constraints statistically. We use this fact to infer the 3D shape of object categories from raw single-view images. Differently from prior work, we use no external supervision, nor do we use multiple views or videos of the objects. We achieve this by a simple reconstruction task, exploiting the symmetry of the objects' shape and albedo. Specifically, given a single image of the object seen from an arbitrary viewpoint, our model predicts a symmetric canonical view, the corresponding 3D shape and a viewpoint transformation, and trains with the goal of reconstructing the input view, resembling an auto-encoder. Our experiments show that this method can recover the 3D shape of human faces, cat faces, and cars from single view images, without supervision. On benchmarks, we demonstrate superior accuracy compared to other methods that use supervision at the level of 2D image correspondences.

2 mentions: @elliottszwu@hiroharu_kato
Date: 2019/06/05 02:18

Referring Tweets

@elliottszwu Our recent work with @chrirupp and Andrea Vedaldi: given only single views of an object category, we learn to predict the 3D shape, lighting, albedo and viewpoint from one raw image, without any other supervision (eg, multiple views, videos, keypoints...). https://t.co/ClZvmgZWzs https://t.co/nIWUmPbY8X
@hiroharu_kato 有名大学だと Stanford と Oxford に使ってもらえてないな…と思ってたら今朝の arXiv に11本目が上がってました。S. Wu, C. Rupprecht, A. Vedaldi (Oxford VGG), "Photo-Geometric Autoencoding to Learn 3D Objects from Unlabelled Images" https://t.co/ABCIzxuJ0l