With advances in machine-learning algorithms and the democratization of big data technologies, machine-learning products have become ubiquitous—they are the de facto choice for powering the experiences that are now common place in many of our beloved apps and services, such as recommendation and personalization. While offline evaluations are routinely done to evaluate algorithm performance, results from live experiments more closely reflects real-world performance and is a necessary step toward a product launch in many companies. However, running experiments on machine-learning products poses unique challenges and requires consideration beyond what traditional experiments frameworks offer.
Grace Huang shares lessons learned from running and interpreting machine-learning experiments and outlines launch considerations that enable sustainable, long-term ecosystem health.
Grace Huang is the data science lead for discovery at Pinterest, where discovery products like recommendations and personalization are developed. She is passionate about building data science products around machine-learning algorithms to drive better experience for Pinterest users and build a sustainable ecosystem.
©2017, O’Reilly UK Ltd • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • firstname.lastname@example.org