Organizers
Sean Fanello, Christoph Rhemann, Graham Fyffe, Jonathan Taylor, Sofien Bouaziz, Rohit Pandey, Paul Debevec, Shahram Izadi
Description
Volumetric (4D) performance capture is fundamental for AR/VR content generation. Designing a volumetric capture pipeline involves developing a full computer vision stack: from novel sensors to efficient volumetric reconstruction algorithms, compression and rendering. To this end, we leverage a combination of active sensors with traditional photometric stereo methods. As a result, we have developed an unique pipeline for reconstruction, tracking and texturing of humans in 4D.
In this tutorial we will walk the attendee through the ins and outs of building such a system from the ground up.
First, we will consider the hardware design choices for cameras, sensors, lighting, and depth estimation algorithms.
In the second part we will cover reconstruction and tracking techniques for people. We will review state of the art algorithms to generate high quality meshes and volumetric representations.
In the third part we will focus on photometric stereo, texturing, relightability and rendering: we will detail the state of the art algorithms together with our choices.
Finally we will discuss some applications and capabilities that 3D capture technologies enable. We will put emphasis on virtual and augmented reality scenarios and highlight recent trends in machine learning that aim at replacing traditional graphics pipelines.
When and Where
June 16th - Long Beach Convention Center in Long Beach, Room 201A
Program
Time |
Title |
Speaker |
|
|---|---|---|---|
9:00 - 9:30 |
[Keynote] Digitizing Humans |
Shahram Izadi |
|
Morning Session: Volumetric Capture of Humans |
|||
9:30 - 10:00 |
High Quality Depth Sensors for Volumetric Capture |
Christoph Rhemann, Adarsh Kowdle |
|
10:00 - 10:30 |
Depth Estimation via Triangulation: Basics, Challenges and Algorithms |
Sergio Orts Escolano |
|
10:30 - 10:45 |
Coffee Break |
||
10:45 - 11:15 |
Depth Estimation in the Age of Deep Learning |
Yinda Zhang, Christian Haene, Sameh Khamis |
|
11:15 - 11:40 |
Multiview Human Pose Estimation via Deep Learning |
Rohit Pandey, Anastasia Tkach |
|
11:40 - 12:30 |
Detect, Reconstruct, Track and Parameterize Humans |
Sean Fanello, Kaiwen Guo |
|
12:30 - 13:30 |
Lunch Break |
||
Afternoon Session: Light Stage for High Quality Renderings |
|||
13:30 - 14:00 |
[Keynote] A Light Stage for (almost) Every Application |
Paul Debevec |
|
14:00 - 14:25 |
The Lightstage Hardware |
Xueming Yu |
|
14:25 - 15:00 |
Relighting Still Images and Videos |
Graham Fyffe |
|
15:00 - 15:20 |
Relightable Volumetric Capture of Humans |
Graham Fyffe |
|
15:20 - 15:40 |
Single Image Portrait Relighting |
Tiancheng Sun UCSD |
15:40 - 16:00 |
Coffee Break |
16:00 - 16:20 |
Neural rendering for Performance Capture |
Rohit Pandey |
|
16:20 - 16:40 |
Neural Rerendering in the Wild |
Moustafa Meshry University of Maryland |
|
16:40 - 17:00 |
Deep Reflectance Fields |
Abhimitra Meka MPI |
Please contact Shahram Izadi or Sean Fanello if you have any questions.