Organizers
Sean Fanello, Federico Tombari, Thabo Beeler, Andrea Colaco
Description
Google Android XR is a new operating system built for the next generation of computing. At the heart of this platform, Computer Vision and Machine Learning are pivotal in ensuring immersive user experiences. In this tutorial, in particular, we will describe how we built from the ground up the full Perception stack: from head tracking algorithms, all the way to photorealistic avatars and scene renderings. Additionally, researchers and engineers will have access to comprehensive references and documentation of the APIs used in this project.
The tutorial begins by emphasizing the significance of data capture, rendering, and groundtruth generation for Perception tasks such as hand, face, and eye tracking.
Next, we explore the construction of an efficient Perception stack, encompassing egocentric head tracking, hand tracking, face tracking, and eye tracking.
Furthermore, we demonstrate how these perception capabilities enable the creation of scalable and efficient photorealistic representations of humans and scenes.
Finally, we showcase use cases and experiences that leverage the full stack, highlighting its potential applications.
When and Where
June 12th - 9.00am CDT - Room 201B, Music City Center, Nashville TN.
Program
| Time | Title | Speaker |
|---|---|---|
| Morning: Sensing and Perception on Android XR | ||
| 9:00-9:15 AM | Intro: Android XR & Project Moohan | Sean Fanello (Google) Sean (Sung Soo) Choi (Samsung) |
| 9:15-9:30 AM | Digitization | Sergio Orts Escolano (Google) Erroll Wood (Google) |
| 9:30-10:00 AM | Foundational Human Models for XR | Stefanos Zafeiriou (Google) |
| 10:00-10:15 AM | Synthetic Data for ML | Erroll Wood (Google) |
| 10:15-10:30 AM | Background for OpenPX and Perception hosting service | Jinshik Bae (Samsung) |
| 10:30-10:45 AM | Real-world scenario of OpenPX - Fit guide in XR | Donghwan Seo (Samsung) |
| 10:45-11:00 AM | Coffee Break | |
| 11:00-11:20 AM | World Tracking for XR | Shuntaro Yamazaki (Google) Abhijeet Bisain (Qualcomm) |
| 11:20-11:45 AM | Hand Tracking for XR | Jonathan Taylor (Google) Abhijeet Bisain (Qualcomm) |
| 11:45 AM-12:10 PM | Face Tracking for XR | Sergio Orts Escolano (Google) |
| 12:10-12:35 PM | Body Tracking for XR | Alexandru-Eugen Ichim (Google) |
| 12:35-1:00 PM | Eye Tracking for XR | Ivana Tosic Rodgers (Google) |
| 1:00-2:00 PM | Lunch Break & Demos | |
| Afternoon: Interaction and Rendering on Android XR | ||
| 2:00-2:30 PM | Video See Through | Eric Turner (Google) Abhijeet Bisain (Qualcomm) |
| 2:30-3:00 PM | Novel View Synthesis for XR | Fabian Manhardt (Google) |
| 3:00-3:30 PM | Open Set 3D Scene Understanding for XR | Federico Tombari (Google) |
| 3:30-3:50 PM | 3D Assets and Immersive Scene Generation for XR | Michael Oechsle (Google) |
| 3:50-4:00 PM | Coffee Break | |
| 4:00-4:30 PM | Scalable Photorealistic Avatars | Yinda Zhang (Google) Yan Deng (Qualcomm) |
| 4:30-5:00 PM | Interactive Perception & Graphics for a Universally Accessible XR | Ruofei Du (Google) |
| 5:00-6:00 PM | Closing Remarks and Demos | |
Please contact Sean Fanello if you have any questions.