Compositing Light Field Video
Using Multiplane Images

Matthew DuVall

John Flynn

Michael Broxton

Paul Debevec

Google Inc.

Click to view the poster.

Publication Links

Poster Abstract PDF.
Abstract PDF.


We present a variety of new compositing techniques using Multiplane Images (MPI’s) derived from footage shot with an inexpensive and portable light field video camera array. The effects include camera stabilization, foreground object removal, synthetic depth of field, and deep compositing.

Traditional compositing is based around layering RGBA images to visually integrate elements into the same scene, and often requires manual 2D and/or 3D artist intervention to achieve realism in the presence of volumetric effects such as smoke or splashing water. We leverage the newly introduced DeepView solver and a light field camera array to generate MPIs stored in the DeepEXR format for compositing with realistic spatial integration and a simple workflow which offers new creative capabilities. We demonstrate using this technique by combining footage that would otherwise be very challenging and time intensive to achieve when using traditional techniques, with minimal artist intervention.

Technical Video

Video demonstrating the virtual camera editing, volume compositing and segmentation as shown in the poster.

Related work

This work was made possible by DeepView View Synthesis with Learned Gradient Descent.