Explore the latest research from Facebook
Filter by
All Publications
October 1, 2021
Hung Le, Chinnadhurai Sankar, Seungwhan Moon, Ahmad Beirami, Alborz Geramifard, Satwik Kottur
DVD: A Diagnostic Dataset for Multi-step Reasoning in Video Grounded Dialogue
The dataset is designed to contain minimal biases and has detailed annotations for the different types of reasoning over the spatio-temporal space of video. Dialogues are synthesized over multiple question turns, each of which is injected with a set of cross-turn semantic relationships. We use DVD to analyze existing approaches, providing interesting insights into their abilities and limitations.
Areas
AR/VR
ARTIFICIAL INTELLIGENCE
Paper
August 23, 2021
Jacob Donley, Vladimir Tourbabin, Boaz Rafaely, Ravish Mehra
Adaptive Multi-Channel Signal Enhancement Based on Multi-Source Contribution Estimation
This paper outlines a new method to adapt to desired and undesired signals using their spatial statistics, independent of their temporal characteristics. The method uses a linearly constrained minimum variance (LCMV) beamformer to estimate the relative source contribution of each source in a mixture, which is then used to weight statistical estimates of the spatial characteristics of each source used for final separation.
Areas
AR/VR
NATURAL LANGUAGE PROCESSING & SPEECH
Paper
August 9, 2021
Sai Bi, Stephen Lombardi, Shunsuke Saito, Tomas Simon, Shih-En Wei, Kevyn Mcphail, Ravi Ramamoorthi, Yaser Sheikh, Jason Saragih
Deep Relightable Appearance Models for Animatable Faces
We present a method for building high-fidelity animatable 3D face models that can be posed and rendered with novel lighting environments in real-time.
Areas
AR/VR
COMPUTATIONAL PHOTOGRAPHY & INTELLIGENT CAMERAS
COMPUTER VISION
Paper
August 9, 2021
Stephen Lombardi, Tomas Simon, Gabriel Schwartz, Michael Zollhoefer, Yaser Sheikh, Jason Saragih
Mixture of Volumetric Primitives for Efficient Neural Rendering
We present Mixture of Volumetric Primitives (MVP), a representation for rendering dynamic 3D content that combines the completeness of volumetric representations with the efficiency of primitive-based rendering, e.g., point-based or mesh-based methods. Our approach achieves this by leveraging spatially shared computation with a convolutional architecture and by minimizing computation in empty regions of space with volumetric primitives that can move to cover only occupied regions.
Areas
AR/VR
COMPUTER VISION
MACHINE LEARNING

Paper
August 9, 2021
Timur Bagautdinov, Chenglei Wu, Tomas Simon, Fabián Prada, Takaaki Shiratori, Shih-En Wei, Weipeng Xu, Yaser Sheikh, Jason Saragih
Driving-Signal Aware Full-Body Avatars
The core intuition behind our method is that better drivability and generalization can be achieved by disentangling the driving signals and remaining generative factors, which are not available during animation.
Areas
AR/VR
COMPUTER VISION

Paper
August 9, 2021
Jungdam Won, Deepak Gopinath, Jessica Hodgins
Control Strategies for Physically Simulated Characters Performing Two-player Competitive Sports
In this paper, we develop a learning framework that generates control policies for physically simulated athletes who have many degrees-of-freedom. Our framework uses a two step-approach, learning basic skills and learning boutlevel strategies, with deep reinforcement learning, which is inspired by the way that people how to learn competitive sports.
Areas
AR/VR
ARTIFICIAL INTELLIGENCE
MACHINE LEARNING
Paper
August 9, 2021
He Zhang, Yuting Ye, Takaaki Shiratori, Taku Komura
ManipNet: Neural Manipulation Synthesis with a Hand-Object Spatial Representation
In this paper, we propose a hand-object spatial representation that can achieve generalization from limited data. Our representation combines the global object shape as voxel occupancies with local geometric details as samples of closest distances. This representation is used by a neural network to regress finger motions from input trajectories of wrists and objects.
Areas
AR/VR
MACHINE LEARNING
Paper
August 6, 2021
Yize Jin, Meixu Chen, Todd Goodall, Anjul Patney, Alan C. Bovik
Subjective and Objective Quality Assessment of 2D and 3D Foveated Video Compression in Virtual Reality
In the 2D study, each video was of resolution 7680×3840 and was viewed and quality-rated by 36 subjects, while in the 3D study, each video was of resolution 5376×5376 and rated by 34 subjects. Both studies were conducted on top of a foveated video player having low motion-to-photon latency (∼50ms).
Areas
AR/VR
Paper
July 7, 2021
Elyse D. Z. Chase, Ali Israr, Pornthep Preechayasomboon, Sarah Sykes, Aakar Gupta, Jessica Hartcher-O’Brien
Learning Vibes: Communication Bandwidth of a Single Wrist-Worn Vibrotactile Actuator
We ran a user study with the salient haptics cues to determine how well people were able to identify them without training on the dorsal side of the wrist, if they could interpret them better with training, and if that knowledge could be transferred to a secondary, untrained location (volar side of the wrist).
Areas
AR/VR
HUMAN COMPUTER INTERACTION & UX
Paper
July 6, 2021
Sonny Chan, Chase Tymms, Nicholas Colonnese
Hasti: Haptic and Audio Synthesis for Texture Interactions
Our method reconstructs meso-and microscopic surface features on the fly along a contact trajectory, and runs a micro-contact dynamics simulation whose outputs drive vibrotactile haptic actuators and modal sound synthesis. An exploratory, absolute identification user study was conducted as an initial evaluation of our synthesis methods.
Areas
AR/VR
Paper
Additional Resources
Videos
Downloads & Projects
Visiting Researchers & Postdocs
Visit Our Other Blogs
Engineering
Facebook AI
Oculus
Tech@
RSS Feed
About
Careers
Privacy
Cookies
Terms
Help
Facebook © 2021
To help personalize content, tailor and measure ads, and provide a safer experience, we use cookies. By clicking or navigating the site, you agree to allow our collection of information on and off Facebook through cookies. Learn more, including about available controls: Cookies Policy