Making data-driven decisions through experimentation is an extremely important part of the culture at LinkedIn. It’s deeply ingrained in the company’s development process and has always been a core part of its DNA. LinkedIn tests everything from complete redesigns of its homepage to changes to the backend relevance algorithm to infrastructure upgrades. It’s how the company innovates, grows, and evolves its products to best serve its members. It’s how LinkedIn makes its members happier, its business stronger, and its talent more productive.
Kapil Surlaker and Ya Xu explain why, to meet the company’s needs, LinkedIn built the UMP and XLNT platforms for metrics computation and experimentation, respectively, which have allowed the company to perform measurement and experimentation very efficiently at scale while preserving trust in data.
The Unified Metrics Platform (UMP) is the source of truth for all metrics at LinkedIn. It ensures the definition of a given metric stays the same independent of where the metric is being used. Users are responsible for authoring the metric logic, and the platform is responsible for executing the logic correctly and efficiently and preserving the SLAs. UMP also detects duplicate metric definitions and alerts the metrics’ authors so there is no redundant computation. A decentralized authoring and review process also means that LinkedIn can be agile while still preserving trust in data.
XLNT allows users to quickly quantify the impact of any A/B test in a scientific and controlled manner across LinkedIn.com and its apps, enabling the easy design and deployment of experiments and providing automatic analysis that is crucial in popularizing A/B tests. The platform is generic and extensible, covering almost all domains, including mobile and email. Every day, hundreds of experiments are run and thousands of metrics computed to improve every aspect of LinkedIn.
Initially, daily metric computation was sufficient to meet most needs, but in many cases, this was far from desirable. During rapid product iterations—the norm at LinkedIn—understanding the impact an experiment has on a product metric soon after enabling it is crucial. In 2016, LinkedIn enhanced the platform to support intraday computation so that users could get reports and monitor the impact of experiments on an hourly basis. More recently, the platform has been further enhanced to enable near-real-time computation, and the metrics have been made available everywhere.
Kapil and Ya offer an overview of both platforms, covering the infrastructure that powers them and the rich visualizations that delight LinkedIn’s users. They also share their experiences of running these platforms in production over the past few years at LinkedIn and explain how they have helped shape the LinkedIn product.
Kapil Surlaker leads the data and analytics team at LinkedIn, where he’s responsible for core analytics infrastructure platforms, including Hadoop, Spark, computation frameworks like Gobblin and Pinot, an OLAP serving store, and XLNT, LinkedIn’s experimentation platform. Previously, Kapil led the development of Databus, a database change capture platform that forms the backbone of LinkedIn’s online data ecosystem; Espresso, a distributed document store that powers many applications on the site; and Helix, a generic cluster management framework that manages multiple infrastructure deployments at LinkedIn. Prior to LinkedIn, Kapil held senior technical leadership positions at Kickfire (acquired by Teradata) and Oracle. Kapil holds a BTech in computer science from IIT Bombay and an MS from the University of Minnesota.
Ya Xu is principal staff engineer and statistician at LinkedIn, where she leads a team of engineers and data scientists building a world-class online A/B testing platform. She also spearheads taking LinkedIn’s A/B testing culture to the next level by evangelizing best practices and pushing for broad-based platform adoption. She holds a PhD in statistics from Stanford University.
Comments on this page are now closed.
©2018, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • email@example.com