Presented By O'Reilly and Cloudera
Make Data Work
22–23 May 2017: Training
23–25 May 2017: Tutorials & Conference
London, UK

Peeking into the black box: Lessons from the front lines of machine-learning product launches

Grace Huang (Pinterest)
10:0510:20 Thursday, 25 May 2017
Location: Auditorium
Average rating: ***..
(3.47, 17 ratings)

With advances in machine-learning algorithms and the democratization of big data technologies, machine-learning products have become ubiquitous—they are the de facto choice for powering the experiences that are now common place in many of our beloved apps and services, such as recommendation and personalization. While offline evaluations are routinely done to evaluate algorithm performance, results from live experiments more closely reflects real-world performance and is a necessary step toward a product launch in many companies. However, running experiments on machine-learning products poses unique challenges and requires consideration beyond what traditional experiments frameworks offer.

Grace Huang shares lessons learned from running and interpreting machine-learning experiments and outlines launch considerations that enable sustainable, long-term ecosystem health.

Photo of Grace Huang

Grace Huang


Grace Huang is the data science lead for discovery at Pinterest, where discovery products like recommendations and personalization are developed. She is passionate about building data science products around machine-learning algorithms to drive better experience for Pinterest users and build a sustainable ecosystem.