Organizers

Sean Fanello, Christoph Rhemann, Graham Fyffe, Jonathan Taylor, Sofien Bouaziz, Paul Debevec, Shahram Izadi

Google

Description

Volumetric (4D) performance capture is fundamental for AR/VR content generation. Designing a volumetric capture pipeline involves developing a full computer vision stack: from novel sensors to efficient volumetric reconstruction algorithms, compression and rendering. To this end, we leverage a combination of active sensors with traditional photometric stereo methods. As a result, we have developed an unique pipeline for reconstruction, tracking and texturing of humans in 4D.

In this tutorial we will walk the attendee through the ins and outs of building such a system from the ground up.

First, we will consider the hardware design choices for cameras, sensors, lighting, and depth estimation algorithms.

In the second part we will cover reconstruction and tracking techniques for people. We will review state of the art algorithms to generate high quality meshes and volumetric representations.

In the third part we will focus on photometric stereo, texturing, relightability and rendering: we will detail the state of the art algorithms together with our choices.

Finally we will discuss some applications and capabilities that 3D capture technologies enable. We will put emphasis on virtual and augmented reality scenarios and highlight recent trends in machine learning that aim at replacing traditional graphics pipelines.

When and Where

June 16th - Long Beach Convention Center in Long Beach, Room 201A

Program

                               
Time
Title
Speaker
9:00 - 9:30
[Keynote] Digitizing Humans
Shahram Izadi
Google
Morning Session:
Volumetric Capture of Humans
9:30 - 10:00
High Quality Depth Sensors for Volumetric Capture
Adarsh Kowdle
Google
10:00 - 10:30
Depth Estimation via Triangulation: Basics, Challenges and Algorithms
Sergio Orts Escolano
Google
10:30 - 10:45
Coffee Break
10:45 - 11:15
Depth Estimation in the Age of Deep Learning
Yinda Zhang, Christian Haene, Sameh Khamis
Google
11:15 - 11:50
Multiview Human Pose Estimation and Segmentation via Deep Learning
Anastasia Tkach
Google
11:50 - 12:30
Non-Rigid Reconstruction, Tracking and Parameterization of Humans
Kaiwen Guo
Google
12:30 - 13:30
Lunch Break
Afternoon Session:
Light Stage for High Quality Renderings
13:30 - 14:00
[Keynote] A Light Stage for (almost) Every Application
Paul Debevec
Google
14:00 - 14:25
The Lightstage Hardware
Xueming Yu
Google
14:25 - 15:00
Relighting Still Images and Videos
Graham Fyffe
Google
15:00 - 15:20
Relightable Volumetric Capture of Humans
Graham Fyffe
Google
15:20 - 15:40
Single Image Portrait Relighting
Tiancheng Sun
UCSD
15:40 - 16:00
Coffee Break
16:00 - 16:20
Compression and Streaming of 4D Performances
Danhang Tang
Google
16:20 - 16:40
Neural rendering for Performance Capture
Rohit Pandey
Google
16:40 - 17:00
Deep Reflectance Fields
Abhimitra Meka
MPI

Please contact Shahram Izadi or Sean Fanello if you have any questions.