Engineering the Future of Software
Feb 3–4, 2019: Training
Feb 4–6, 2019: Tutorials & Conference
New York, NY
Please log in

Architecting for news recommendations: The many hidden colors of the Gray Lady

Aikaterini Iliakopoulou (The New York Times)
3:50pm–4:40pm Tuesday, February 5, 2019
Application architecture
Location: Trianon Ballroom
Secondary topics:  Case Study
Average rating: ****.
(4.64, 14 ratings)

Who is this presentation for?

  • Machine learning engineers, software architects, and software developers



Prerequisite knowledge

  • Familiarity with microservices, A/B testing, and machine learning (useful but not required)

What you'll learn

  • Explore the architecture solutions employed by the New York Times for building pipelines for recommender systems
  • Gain insights into the business rules that apply when personalizing for news, how these rules translate to architectural patterns, and the benefits and challenges they offer


When personalizing for news platforms, you must pay attention to both the audience and the content. Much like most software architecture systems, readers and news content should not be treated as a monolith. There are many types of audiences, each with a set of unique preferences and habits. At the same time, most news organizations nowadays produce a variety of content, addressing different audiences and different interests.

The New York Times produces content for 63 different sections, and many of those sections feature diverse content. It’s in the company’s interest to surface the right content to the right audience by personalizing the user experience on our platforms. To do so, it has built a set of recommender models, from the generic and simple, such as ranking by popularity or a Thompson sampling model, to the complex and personalized, such as collaborative filtering using matrix factorization and contextual bandits.

The New York Times’s strategy to understand its readers and how they engage with its platforms is to A/B test different models on different types of content, such as the Opinion block on the homepage or the Smarter Living section on the iOS app. This requires an architecture that allows the company to easily modify an experiment on its platforms for a surface and add models that look more promising while easily removing the ones that perform poorly. One challenging business requirement is to leave some editorial control over how content is personalized to the newsroom’s expertise to make sure what New York Times shows on its platforms aligns with what they deem to be most important for readers to see (for example, a great investigative piece) or not see (an article that’s not relevant anymore, such as a Thanksgiving recipe).

Katerina Iliakopoulou details the architecture of a system that facilitates agile personalization on the New York Times’s platforms to accommodate fast A/B testing, discussing software architecture patterns the company tried and the solutions that worked and allowed for fast model integration and use on any pool of content with a different set of business rules. Along the way, Katerina touches on scalability problems encountered (and solutions) and explains how the current modular architecture solution allows for service extensibility as the company moves to expand its recommender models.

Photo of Aikaterini Iliakopoulou

Aikaterini Iliakopoulou

The New York Times

Katerina Iliakopoulou is a senior software engineer in the Messaging Group at the New York Times, where she maintains the platform that delivers newsroom produced messages, being either a newsletter or a push notification. She’s interested in machine learning, information retrieval, and system design—especially if they’re applied in a news media context. In her three years at the New York Times, she’s led the architectural design of one of the team’s main frameworks for serving recommendations and worked on a variety of projects that facilitate the use of recommendations on the New York Times’s platforms, either by building microservices or collaborating with data scientists to productionalize recommender models.