Single-view view synthesis with multiplane images
CVPR 2020

Richard Tucker
Noah Snavely
Google Research


Single image inputMultiplane imageNovel rendered views

Abstract

A recent strand of work in view synthesis uses deep learning to generate multiplane images—a camera-centric, layered 3D representation—given two or more input images at known viewpoints. We apply this representation to single-view view synthesis, a problem which is more challenging but has potentially much wider application. Our method learns to produce a multiplane image directly from a single image input, predicting shape and disoccluded content in a single step, and we introduce scale-invariant view synthesis for supervision, enabling us to train on online video. We show this approach is applicable to several different datasets, that it additionally generates reasonable depth maps, and that it learns to fill in content behind the edges of foreground objects in background layers.

Paper

Video

Interactive examples

Click these images to display the predicted MPIs in an interactive viewer.
(Images used under Creative Commons license from YouTube user SonaVisual.)

BibTeX

@InProceedings{single_view_mpi, author = {Tucker, Richard and Snavely, Noah}, title = {Single-view View Synthesis with Multiplane Images}, booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)}, month = {June}, year = {2020} }