In 2016, Tribune Publishing began built an in-house data science team to better leverage its vast datasets with new machine learning and analytics technologies. One of the primary successes of this team was its content recommendation system (“RecSys”), developed entirely in house on top of existing open source systems and new open source libraries created and released by Tribune.
Requirements for the RecSys included the ability to perform A/B/n testing against legacy human-edited and algorithmic recommendations, support multiple publications with both shared and exclusive content, support “real-time” online machine learning at scale, scale without limit in the face of traffic spikes, and gracefully degrade when responses can’t be delivered within a given time limit.
Matt Chapman leads a walkthrough of the lifecycle of the request from the web browser of a news-reading end user to the backend algorithms that generates up-to-the moment, personalized recommendations for what the user might want to read next. Along the way, Matt reviews the challenges that the team faced, the open source solutions used at each step, and the new framework and libraries developed by the team to make development of algorithms and of the system itself fast, flexible, and scalable.
Matt Chapman is manager of data engineering at mPulse Mobile. Previously, he was the lead data engineer for Tribune Interactive. A hands-on leader of software engineering, Matt has professional programming experience in at least eight languages, many databases, and many more frameworks. He’s formed and contributed to teams of technologists for companies in a broad spectrum of industries, including web publishing, real estate, event management, finance, media, and healthcare, most recently focusing on applications of big data, machine learning, and data science.
©2019, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • email@example.com