Jointly Optimize Capacity, Latency and Engagement in Large-scale Recommendation Systems
The ACM Conference on Recommender Systems (RecSys)
September 27, 2021
By: Hitesh Khandelwal, Viet Ha-Thuc, Avishek Dutta, Yining Lu, Nan Du, Zhihao Li, Qi Huang
As the recommendation systems behind commercial services scale up and apply more and more sophisticated machine learning models, it becomes important to optimize computational cost (capacity) and runtime latency, besides the traditional objective of user engagement. Caching recommended results and reusing them later is a common technique used to reduce capacity and latency. However, the standard caching approach negatively impacts user engagement. To overcome the challenge, this paper presents an approach to optimizing capacity, latency and engagement simultaneously. We propose a smart caching system including a lightweight adjuster model to refresh the cached ranking scores, achieving significant capacity savings without impacting ranking quality. To further optimize latency, we introduce a prefetching strategy which leverages the smart cache. Our production deployment on Facebook Marketplace demonstrates that the approach reduces capacity demand by 50% and p75 end-to-end latency by 35%. While Facebook Marketplace is used as a case study, the approach is applicable to other industrial recommendation systems as well.
Download Paper
Related Publications
Interspeech - October 12, 2021
LiRA: Learning Visual Speech Representations from Audio through Self-supervision
Pingchuan Ma, Rodrigo Mira, Stavros Petridis, Björn W. Schuller, Maja Pantic
ICML - July 18, 2021
Latency-Aware Neural Architecture Search with Multi-Objective Bayesian Optimization
David Eriksson, Pierce I-Jen Chuang, Samuel Daulton, Peng Xia, Akshat Shrivastava, Arun Babu, Shicong Zhao, Ahmed Aly, Ganesh Venkatesh, Maximilian Balandat
ISAAC - December 5, 2021
On the Extended TSP Problem
Julián Mestre, Sergey Pupyrev, Seeun William Umboh
IEEE Transactions on Image Processing Journal - March 9, 2021
Inspirational Adversarial Image Generation
Baptiste Rozière, Morgane Rivière, Olivier Teytaud, Jérémy Rapin, Yann LeCun, Camille Couprie
All Publications
Additional Resources
Downloads & Projects
Visiting Researchers & Postdocs
Visit Our Other Blogs
Facebook AI
RSS Feed
Facebook © 2021
To help personalize content, tailor and measure ads, and provide a safer experience, we use cookies. By clicking or navigating the site, you agree to allow our collection of information on and off Facebook through cookies. Learn more, including about available controls: Cookies Policy