14–17 Oct 2019

Herding cats: Product management in the machine learning era

Ira Cohen (Anodot), Arun Kejariwal (Independent)
9:0012:30 Tuesday, 15 October 2019
Location: Buckingham Room - Palace Suite
Secondary topics:  Machine Learning
Average rating: ***..
(3.25, 8 ratings)

Who is this presentation for?

  • Product managers, data science managers, and experienced data science architects

Level

Beginner

Description

While the role of the manager doesn’t require deep knowledge of machine learning algorithms, it does require understanding how ML-based products should be developed and the ability to bridge the gap between product and business requirements and the inherent uncertainty at any ML-based solution. This uncertainty follows an ML-based solution at all phases: there’s rarely certainty that any ML-based solution can solve a given business problem.

The development cycle involves research iterations to continuously try improving the results—a cycle that isn’t deterministic. After the deployment of a solution, the results aren’t deterministic and may be erroneous, requiring handling of those in the product design and also careful monitoring of the ML performance (e.g., a model predicting churn of a subscriber outputs a probability for churn, which by itself may be wrong because of modeling issues, input data issues, etc.).

Ira Cohen walks you through the cycle of developing ML-based capabilities (or entire products) and the role of the (product) manager in each step of the cycle using real-world examples.

Outline:

Initial design

  • Mapping a business problem to a ML solution (e.g., type of ML solution required, any constraints on learning time, runtime, and more)
  • Mapping the outcomes of ML to the desired outcome from the user perspective (e.g., accuracy, visualization, and product flow wrapping the ML capability)

Research phase

  • Data collection: Determining which data is available or should be produced
  • Experimentation: Determining the success criteria
  • How to know when to stop iterations

Deployment phase

  • Requirements from the production process (real time versus offline, batch operations versus stream operations, etc.)
  • How to roll out new versions and improvements

Production phase

  • Monitoring requirements to ensure ML-based service performance
  • Product design to handle uncertainty in outcomes of the ML component

Prerequisite knowledge

  • A basic knowledge of product management

Materials or downloads needed in advance

  • A laptop

What you'll learn

  • Learn the key factors to being a successful product manager of machine learning based products
Photo of Ira Cohen

Ira Cohen

Anodot

Ira Cohen is a cofounder and chief data scientist at Anodot, where he’s responsible for developing and inventing the company’s real-time multivariate anomaly detection algorithms that work with millions of time series signals. He holds a PhD in machine learning from the University of Illinois at Urbana-Champaign and has over 12 years of industry experience.

Photo of Arun Kejariwal

Arun Kejariwal

Independent

Arun Kejariwal is an independent lead engineer. Previously, he was he was a statistical learning principal at Machine Zone (MZ), where he led a team of top-tier researchers and worked on research and development of novel techniques for install-and-click fraud detection and assessing the efficacy of TV campaigns and optimization of marketing campaigns, and his team built novel methods for bot detection, intrusion detection, and real-time anomaly detection; and he developed and open-sourced techniques for anomaly detection and breakout detection at Twitter. His research includes the development of practical and statistically rigorous techniques and methodologies to deliver high performance, availability, and scalability in large-scale distributed clusters. Some of the techniques he helped develop have been presented at international conferences and published in peer-reviewed journals.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)

Contact us

confreg@oreilly.com

For conference registration information and customer service

partners@oreilly.com

For more information on community discounts and trade opportunities with O’Reilly conferences

aisponsorships@oreilly.com

For information on exhibiting or sponsoring a conference

pr@oreilly.com

For media/analyst press inquires