Getting the most out of your AI projects with model feedback loops
Who is this presentation for?
- Data scientists, machine learning engineers, and data science team managers
Building machine learning models consumes a considerable amount of time, money and energy. But much of the investment is lost if your model performance isn’t measured and monitored. While many people talk about how to deploy a model to production, very few discussions cover how to measure the model’s performance. Hannes Hapke and Catherine Nelson want to change that by introducing model feedback loops.
Model feedback loops have two important purposes: measure the model’s performance and increase the model’s underlying data set for future model updates.
You’ll explore how you can set up feedback loops, measure your model’s performance, and how to use feedback loops to retrain you models. They introduce metrics to measure your model’s performance, highlight user experience considerations when designing model feedback loops, and outline a framework for building reproducible feedback loops.
- Familiarity with basic machine learning concepts and at least one machine learning framework such as PyTorch, TensorFlow, or Keras (The examples are based on TensorFlow and Keras, but core concepts can be applied to any framework.)
What you'll learn
- Discover what feedback loops are in the context of machine learning, why they're important, and what UX considerations should be applied when designing feedback loops
- Learn how to measure a model’s performance, the ROI of the data science project, and which KPIs to choose; how explicit and implicit feedback differ; and how to efficiently track your model’s performance and how to turn it into new datasets to generate new model versions
Hannes Hapke is a senior data scientist at SAP ConcurLabs. He’s been a machine learning enthusiast for many years and is a Google Developer Expert for machine learning. Hannes has applied deep learning to a variety of computer vision and natural language problems, but his main interest is in machine learning infrastructure and automating model workflows. Hannes is a coauthor of the deep learning publication Natural Language Processing in Action and he’s currently working on a book about TensorFlow Extended Building Machine Learning Pipelines (O’Reilly). When he isn’t working on a deep learning project, you’ll find him outdoors running, hiking, or enjoying a good cup of coffee with a great book.
Concur Labs, SAP Concur
Catherine Nelson is a senior data scientist for Concur Labs at SAP Concur, where she explores innovative ways to use machine learning to improve the experience of a business traveller. She’s particularly interested in privacy-preserving ML and applying deep learning to enterprise data. Previously, she was a geophysicist and studied ancient volcanoes and explored for oil in Greenland. Catherine has a PhD in geophysics from Durham University and a master’s of earth sciences from Oxford University.
Leave a Comment or Question
Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?
Join the conversation here (requires login)
Premier Diamond Sponsors
Premier Exhibitor Plus
For conference registration information and customer service
For more information on community discounts and trade opportunities with O’Reilly conferences
For information on exhibiting or sponsoring a conference
For media/analyst press inquires