Most applications of machine learning across science and industry rely on the holdout method for model selection and validation. Unfortunately, the holdout method can fail in the now common situation
where the data scientist works interactively with the data, iteratively choosing which methods to use by probing the same holdout data many times.
Moritz Hardt outlines a reusable holdout method, which can be used many times without losing the guarantees of fresh data. Moritz also explains how to design reliable machine-learning benchmarks for a number of applications such as data science competitions and hyperparameter tuning.
Moritz Hardt is a senior research scientist at Google Research, where his mission is to build the theory and tools that make machine learning more reliable. After obtaining a PhD in computer science from Princeton University, Moritz spent three years at IBM Research Almaden prior to joining Google.
©2016, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • email@example.com
Apache Hadoop, Hadoop, Apache Spark, Spark, and Apache are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries, and are used with permission. The Apache Software Foundation has no affiliation with and does not endorse, or review the materials provided at this event, which is managed by O'Reilly Media and/or Cloudera.