VoLux-GAN: A Generative Model for 3D Face Synthesis with HDRI Relighting
Feitong Tan1,2 Sean Fanello1 Abhimitra Meka1 Sergio Orts-Escolano1 Danhang Tang1 Rohit Pandey1
Jonathan Taylor1 Ping Tan2 Yinda Zhang1
1 Google 2 Simon Fraser University
We propose VoLux-GAN, a generative framework to synthesize 3D-aware faces with convincing relighting. Our main contribution is a volumetric HDRI relighting method that can efficiently accumulate albedo, diffuse and specular lighting contributions along each 3D ray for any desired HDR environmental map. Additionally, we show the importance of supervising the image decomposition process using multiple discriminators. In particular, we propose a data augmentation technique that leverages recent advances in single image portrait relighting to enforce consistent geometry, albedo, diffuse and specular components. Multiple experiments and comparisons with other generative frameworks show how our model is a step forward towards photorealistic relightable generative models.
@misc{tan2022voluxgan,
title={VoLux-GAN: A Generative Model for 3D Face Synthesis with HDRI Relighting},
author={Feitong Tan and Sean Fanello and Abhimitra Meka and Sergio Orts-Escolano and Danhang Tang and Rohit Pandey and Jonathan Taylor and Ping Tan and Yinda Zhang},
year={2022},
eprint={2201.04873},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
