Driving-Signal Aware Full-Body Avatars
August 9, 2021
By: Timur Bagautdinov, Chenglei Wu, Tomas Simon, Fabián Prada, Takaaki Shiratori, Shih-En Wei, Weipeng Xu, Yaser Sheikh, Jason Saragih
We present a learning-based method for building driving-signal aware full-body avatars. Our model is a conditional variational autoencoder that can be animated with incomplete driving signals, such as human pose and facial keypoints, and produces a high-quality representation of human geometry and view-dependent appearance. The core intuition behind our method is that better drivability and generalization can be achieved by disentangling the driving signals and remaining generative factors, which are not available during animation. To this end, we explicitly account for information deficiency in the driving signal by introducing a latent space that exclusively captures the remaining information, thus enabling the imputation of the missing factors required during full-body animation, while remaining faithful to the driving signal. We also propose a learnable localized compression for the driving signal which promotes better generalization, and helps minimize the influence of global chance-correlations often found in real datasets. For a given driving signal, the resulting variational model produces a compact space of uncertainty for missing factors that allows for an imputation strategy best suited to a particular application. We demonstrate the efficacy of our approach on the challenging problem of full-body animation for virtual telepresence with driving signals acquired from minimal sensors placed in the environment and mounted on a VR-headset.
Download Paper
Related Publications
ICCV - October 11, 2021
Neural-GIF: Neural Generalized Implicit Functions for Animating People in Clothing
Garvita Tiwari, Nikolaos Sarafianos, Tony Tung, Gerard Pons-Moll
ICCV - October 11, 2021
Worldsheet: Wrapping the World in a 3D Sheet for View Synthesis from a Single Image
Ronghang Hu, Nikhila Ravi, Alexander C. Berg, Deepak Pathak
SIGGRAPH - August 2, 2021
Fast Diffraction Pathfinding for Dynamic Sound Propagation
Carl Schissler, Gregor Mückl, Paul Calamia
CVPR - June 21, 2021
img2pose: Face Alignment and Detection via 6DoF, Face Pose Estimation
Vítor Albiero, Xingyu Chen, Xi Yin, Guan Pang, Tal Hassner
All Publications
Additional Resources
Downloads & Projects
Visiting Researchers & Postdocs
Visit Our Other Blogs
Facebook AI
RSS Feed
Facebook © 2021
To help personalize content, tailor and measure ads, and provide a safer experience, we use cookies. By clicking or navigating the site, you agree to allow our collection of information on and off Facebook through cookies. Learn more, including about available controls: Cookies Policy