Sep 23–26, 2019

Building a Machine Learning Framework to Measure TV Advertising Attribution

Fei Wang (CarGurus), Michael Brautbar (CarGurus)
2:55pm3:35pm Wednesday, September 25, 2019
Location: 1A 08/10
Secondary topics:  Media and Advertising, Retail and e-commerce

Who is this presentation for?

Senior data scientist / Statistician




In the online automotive shopping space one of the most important yet undoubtedly complex measurements is lead attribution. Companies investing in their brand marketing via Digital Performance Marketing, TV advertising, content marketing, etc., need to measure the cost effectiveness of spend for each medium and each campaign. However, advertisers are facing a challenge to measure TV’s attribution using non-randomized and non-experimental data when digital performance marketing and digital content advertising are running at the same time.

In the case of CarGurus, until June 2017, traffic acquisition was driven solely by a proprietary algorithmic digital performance marketing model. In June 2017, CarGurus launched its first national television advertising campaign in the U.S. The Data Science team was tasked with building a new TV attribution model using a causal inference framework to measure uniquely TV driven leads and then calculated how effective the campaign was - the cost per acquisition (CPA) metric.

In this talk, we present a novel machine learning tool based on the state space model to quantify casual effects of TV, digital performance marketing and digital content advertising simultaneously. We emphasize the conceptual framework to explicitly include potential and important causal factors driving leads, and interpretability of the causal inference model.

Our approach is two-fold. First, by using historical lead data during the pre-test period, we build a state space model to understand the data generation process before TV advertising. This model is then used to predict the counterfactual leads during the post-test period. The difference between actual and predicted counterfactual leads is treated as the incremental leads not from digital performance marketing (non-DPM). This non-DPM contribution not only contains the causal effect from the TV spend but may also contain contributions from other factors, for example improved mobile website design and increased vehicle listings on CarGurus website during the post-test period. Second, causal effect from the TV ads is obtained by stripping out contributions from other factors except TV ads using certain domain knowledge.

By attending this session, you’ll experience a deeper dive into the causal inference model including:

• The state space model with exogenous variables to account for the inherent lack of control groups and lack of randomization with TV advertising campaigns
• How to correct for non-TV factors and simultaneously obtain causal effects for TV, digital performance marketing, and digital content advertising
• How to measure TV’s lagged effects and some statistical techniques to improve model interpretability
• Understanding the interplay of TV advertising and digital performance marketing: how to attribute part of your digital performance marketing costs to TV advertising origin.
CPA calculation by identifying extra digital performance marketing cost

Prerequisite knowledge

Time series analysis (optional)

What you'll learn

Attendees of the session will walk away with a blueprint for addressing the complexities of modelling attribution within a multi-touchpoint marketing strategy. These insights will help to improve your understanding of spend effectiveness across mediums and specific campaigns.
Photo of Fei Wang

Fei Wang


Fei Wang is a senior data scientist/Statistician at CarGurus. His work primarily involves experimental design, causal inference modeling for
online and TV advertising. Fei’s research includes statistical machine learning, matrix factorization, optimization and high-dimensional data modeling. Fei holds a Ph.D in Biostatistics from University of Michigan.

Photo of Michael Brautbar

Michael Brautbar


Michael leads the data science team at CarGurus, a prominent automotive marketplace. His expertise is in algorithms, data mining, deep learning and machine learning, and statistics. His work focuses on creating solutions to central problems in online advertising and other fundamental aspects of the business. In the past, he worked in the capacities of director of data science, lead data scientist and principal data scientist, applying his expertise to create solutions to prime problems in actuary and healthcare, CRM, restaurant management software, and retail.

Michael is a Hebrew University alumnus (BS, MS), a University of Pennsylvania alumnus (MS, PhD), and a Microsoft Research alumnus.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)

Contact us

For conference registration information and customer service

For more information on community discounts and trade opportunities with O’Reilly conferences

For information on exhibiting or sponsoring a conference

Contact list

View a complete list of Strata Data Conference contacts