Explore the latest research from Facebook
Filter by
All Publications
December 13, 2021
Vimal Manohar, Tatiana Likhomanenko, Qiantong Xu, Wei-Ning Hsu, Ronan Collobert, Yatharth Saraf, Geoffrey Zweig, Abdelrahman Mohamed
Kaizen: Continuously Improving Teacher Using Exponential Moving Average For Semi-supervised Speech Recognition
In this paper, we introduce the Kaizen framework that uses a continuously improving teacher to generate pseudo-labels for semi-supervised speech recognition (ASR). The proposed approach uses a teacher model which is updated as the exponential moving average (EMA) of the student model parameters.
Areas
NATURAL LANGUAGE PROCESSING & SPEECH
Paper
November 10, 2021
Marzieh Saeidi, Majid Yazdani, Andreas Vlachos
Cross-Policy Compliance Detection via Question Answering
In this paper we propose to address policy compliance detection via decomposing it into question answering, where questions check whether the conditions stated in the policy apply to the scenario, and an expression tree combines the answers to obtain the label. Despite the initial upfront annotation cost, we demonstrate that this approach results in better accuracy, especially in the cross-policy setup where the policies during testing are unseen in training.
Areas
NATURAL LANGUAGE PROCESSING & SPEECH
Paper
November 9, 2021
Verna Dankers, Anna Langedijk, Kate McCurdy, Adina Williams, Dieuwke Hupkes
Generalising to German Plural Noun Classes, from the Perspective of a Recurrent Neural Network
Here, in line with that tradition, we explore how recurrent neural networks acquire the complex German plural system and reflect upon how their strategy compares to human generalisation and rule-based models of this system.
Areas
ARTIFICIAL INTELLIGENCE
NATURAL LANGUAGE PROCESSING & SPEECH
Paper
November 7, 2021
Shuo Sun, Ahmed El-Kishky, Vishrav Chaudhary, James Cross, Francisco Guzmán, Lucia Specia
Classification-based Quality Estimation: Small and Efficient Models for Real-world Applications
In this work, we evaluate several model compression techniques for QE and find that, despite their popularity in other NLP tasks, they lead to poor performance in this regression setting.
Areas
NATURAL LANGUAGE PROCESSING & SPEECH
Paper
October 31, 2021
Pedro Rodriguez, Jordan Boyd-Graber
Evaluation Paradigms in Question Answering
This position paper names and distinguishes these paradigms. Despite substantial overlap, subtle but significant distinctions exert an outsize influence on research. While one evaluation paradigm values creating more intelligent QA systems, the other paradigm values building QA systems that appeal to users.
Areas
ARTIFICIAL INTELLIGENCE
NATURAL LANGUAGE PROCESSING & SPEECH
Paper
October 11, 2021
Yash Kant, Abhinav Moudgil, Dhruv Batra, Devi Parikh, Harsh Agrawal
Contrast and Classify: Training Robust VQA Models
We propose a novel training paradigm (ConClaT) that optimizes both cross-entropy and contrastive losses. The contrastive loss encourages representations to be robust to linguistic variations in questions while the cross-entropy loss preserves the discriminative power of representations for answer prediction.
Areas
ARTIFICIAL INTELLIGENCE
COMPUTER VISION
MACHINE LEARNING
NATURAL LANGUAGE PROCESSING & SPEECH
Paper
October 3, 2021
Yihong Chen, Pasquale Minervini, Sebastian Riedel, Pontus Stenetorp
Relation Prediction as an Auxiliary Training Objective for Improving Multi-Relational Graph Representations
In this paper, we propose a new self-supervised training objective for multi-relational graph representation learning, via simply incorporating relation prediction into the commonly used 1vsAll objective.
Areas
NATURAL LANGUAGE PROCESSING & SPEECH
Paper
September 3, 2021
Adam Polyak, Yossi Adi, Jade Copet, Eugene Kharitonov, Kushal Lakhotia, Wei-Ning Hsu, Abdelrahman Mohamed, Emmanuel Dupoux
Speech Resynthesis from Discrete Disentangled Self-Supervised Representations
We propose using self-supervised discrete representations for the task of speech resynthesis. To generate disentangled representation, we separately extract low-bitrate representations for speech content, prosodic information, and speaker identity.
Areas
ARTIFICIAL INTELLIGENCE
MACHINE LEARNING
NATURAL LANGUAGE PROCESSING & SPEECH
Paper
August 31, 2021
Tatiana Likhomanenko, Qiantong Xu, Jacob Kahn, Gabriel Synnaeve, Ronan Collobert
slimIPL: Language-Model-Free Iterative Pseudo-Labeling
We improve upon the IPL algorithm: as the model learns, we propose to iteratively re-generate transcriptions with hard labels (the most probable tokens), that is, without a language model. We call this approach Language-Model-Free IPL (slimIPL) and give a resultant training setup for low-resource settings with CTC-based models. slimIPL features a dynamic cache for pseudo-labels which reduces sensitivity to changes in relabeling hyperparameters and results in improved training stability.
Areas
NATURAL LANGUAGE PROCESSING & SPEECH
Paper
August 30, 2021
Ju Lin, Yun Wang, Kaustubh Kalgaonkar, Gil Keren, Didi Zhang, Christian Fuegen
A Two-stage Approach to Speech Bandwidth Extension
In this paper, we propose a two-stage approach for BWE, which enjoys the advantages of both time- and frequency-domain methods.
Areas
NATURAL LANGUAGE PROCESSING & SPEECH
Paper
Additional Resources
Videos
Downloads & Projects
Visiting Researchers & Postdocs
Visit Our Other Blogs
Engineering
Facebook AI
Oculus
Tech@
RSS Feed
About
Careers
Privacy
Cookies
Terms
Help
Facebook © 2021
To help personalize content, tailor and measure ads, and provide a safer experience, we use cookies. By clicking or navigating the site, you agree to allow our collection of information on and off Facebook through cookies. Learn more, including about available controls: Cookies Policy