“Ten lessons learned” by Dr. Xavier Amatriain

Netflix machine learning master Dr. Xavier Amatriain gave a very thoughtful keynote presentation at the 2014 MLConf: “Ten Lessons Learned from Building (real-life impactful) Machine Learning Systems“. These ten lessons are (with elucidations and/or my additions):

  1. “More data vs. and better models/algorithms” (The “vs.” implies a needless dichotomy.)
  2. “You might not need all your Big Data” (How do your features relate to the problem you want to solve?)
  3. “The fact that a more complex model does not improve things does not mean you don’t need one” (Do not treat as independent the features and model, but optimise together.)
  4. “Be thoughtful about how you define your training/testing data sets” (Assembling the material of an experiment is far more complex task than it first appears.)
  5. “Learn to deal with (the curse of) the Presentation Bias” (All sorts of validity traps exist in the design, implementation and analysis of an evaluation.)
  6. “The UI is the only communication channel between the algorithm and what matters most: the Users” (Not only must you think about the machine, but also the end user.)
  7. “Data and models are great. You know what is even better? The right evaluation approach” (The importance of getting the evaluation correct (valid and relevant) cannot be understated. Evaluation is not simply choosing a dataset and a metric.)
  8. “Distributing algorithms is important. Knowing at what level to distribute them is even more so” (Taking systems into the real world brings new and important problems.)
  9. “It pays off to be smart about choosing your hyperparameters” (Grid search can be groping around in a dark dynamic environment.)
  10. “There are things you can do offline and there are things you can’t… and there is Nearline for everything in between” (Going from algorithms to usable systems is another engineering problem.)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s