Now Playing
The Facebook Field Guide to…
8:14
Field Guide to Machine Learning,…
9:32
The Field Guide to Machine…
10:05
The Field Guide to Machine…
10:46
The Field Guide to Machine…
Machine Learning Academy
The Facebook Field Guide to Machine Learning, Episode 6: Experimentation
May 22, 2018
The Facebook Field Guide to Machine Learning is a six-part video series developed by the Facebook ads machine learning team. The series shares best real-world practices and provides practical tips about how to apply machine-learning capabilities to real-world problems.
If you’re interested in using machine learning to enhance your product in the real world, it’s important to understand how the entire development process works. It’s not only what happens during the training of your models, but everything that comes before and after, and how each step can either set you up for success or doom you to fail.
The Facebook ads machine learning team has developed a series of videos to help engineers and new researchers learn to apply their machine learning skills to real-world problems. The Facebook Field Guide to Machine Learning series breaks down the machine learning process into six steps:
  1. Problem definition
  2. Data
  3. Evaluation
  4. Features
  5. Model
  6. Experimentation
This video series covers each of these steps, explaining how the decisions you make along the way can help you successfully apply machine learning to your product or use case. Each lesson highlights examples and stories of non-obvious things that can be important in an applied setting.
Lesson 6: Experimentation
In this last lesson of the Facebook Field Guide to Machine Learning series, we cover experimentation, or making your experiments actionable. One of the key topics is the difference between offline and online experimentation.
Offline experimentation is a fast and reproducible way of testing a machine learning system in isolation. For some image and speech recognition applications, an offline evaluation is representative of using those models in real life. In most cases, however, this is not the case. However, online experimentation allows us to see how our model performs when deployed into the engineering stack, and also when interacting with the world around it.
There are few things more demoralizing than pouring yourself into a difficult project and getting great offline performance before finally testing it online and seeing that that things aren’t working the way you thought they would.
In this lesson, we share a few key principles for effective online testing:
  1. Minimize the time to first online experiment
  2. Isolate engineering bugs from machine learning performance
  3. Test the model in the presence of real-world feedback loops
Some additional practical tips covered in this video include:
Measure the right thing
Measure everything
Be able to triangulate each change
Have a backup plan
Calibrate – measure how well the average prediction matches the average response rate
Thank you for joining us for the Facebook Field Guide to Machine Learning, we hope these lessons are useful as you think about edge cases, and how machine learning models fit into the product or business outcome you are trying to achieve.
 
Areas
MACHINE LEARNING
Share
All The Latest
Filter by
Series
October 14, 2020
Developing tools, technologies and expertise to evolve the way we design
October 14, 2020
Innovation comes through exploration and challenging our assumptions
September 8, 2020
Bringing together different functions to find new perspective and achieve design excellence
August 31, 2020
TexMesh: Reconstructing Human Texture and Geometry from Monocular Video
August 31, 2020
Are Labels Necessary for Neural Architecture Search?
August 31, 2020
TextCaps: a Dataset for Image Captioning with Reading Comprehension
August 24, 2020
Giving people the support and autonomy they need
August 24, 2020
Improving Vision-and-Language Navigation with Web Image-Text Pairs
August 21, 2020
SoundSpaces: Audio-Visual Navigation in 3D Environments
Additional Resources
Videos
Downloads & Projects
Visiting Researchers & Postdocs
Visit Our Other Blogs
Engineering
Facebook AI
Oculus
Tech@
RSS Feed
About
Careers
Privacy
Cookies
Terms
Help
Facebook © 2021
To help personalize content, tailor and measure ads, and provide a safer experience, we use cookies. By clicking or navigating the site, you agree to allow our collection of information on and off Facebook through cookies. Learn more, including about available controls: Cookies Policy