Sep 23–26, 2019

Building a machine learning framework to measure TV advertising attribution

Fei Wang (CarGurus)
2:55pm3:35pm Wednesday, September 25, 2019
Location: 1A 08/10

Who is this presentation for?

  • Senior data scientists and statisticians

Level

Intermediate

Description

In the online automotive shopping space, one of the most important, yet undoubtedly complex measurements, is lead attribution. Companies investing in their brand marketing via digital performance marketing, TV advertising, content marketing, etc., need to measure the cost effectiveness of spend for each medium and each campaign. However, advertisers face a challenge in measuring TV’s attribution using nonrandomized and nonexperimental data when digital performance marketing and digital content advertising are running at the same time.

In the case of CarGurus, traffic acquisition had been driven solely by a proprietary algorithmic digital performance marketing model. In June 2017, CarGurus launched its first national television advertising campaign in the US. The data science team was tasked with building a new TV attribution model using a causal inference framework to measure uniquely TV-driven leads and calculate how effective the campaign was—the CPA metric.

Fei Wang takes a deep dive into a novel machine learning tool based on the state space model to quantify casual effects of TV, digital performance marketing, and digital content advertising simultaneously, emphasizing the conceptual framework to explicitly include potential and important causal factors driving leads and interpretability of the causal inference model.

The approach is two-fold. First, by using historical lead data during the pretest period, CarGurus built a state space model to understand the data-generation process before TV advertising. This model then predicts the counterfactual leads during the posttest period. The difference between actual and predicted counterfactual leads is treated as the incremental leads not from digital performance marketing (non-DPM). This non-DPM contribution not only contains the causal effect from the TV spend but may also contain contributions from other factors, for example, improved mobile website design and increased vehicle listings on the CarGurus website during the posttest period. Second, causal effect from the TV ads is obtained by stripping out contributions from factors except TV ads using certain domain knowledge.

You’ll leave having taken a deep dive into the causal inference model, including the state space model with exogenous variables to account for the inherent lack of control groups and lack of randomization with TV advertising campaigns; how to correct for non-TV factors and simultaneously obtain causal effects for TV, digital performance marketing, and digital content advertising; how to measure TV’s lagged effects and some statistical techniques to improve model interpretability; the interplay of TV advertising and digital performance marketing and how to attribute part of your digital performance marketing costs to TV advertising origin; and CPA calculation by identifying extra digital performance marketing cost.

Prerequisite knowledge

  • A basic understanding of time series analysis (useful but not required)

What you'll learn

  • Discover a blueprint for addressing the complexities of modeling attribution within a multitouchpoint marketing strategy
  • Understand spend effectiveness across mediums and specific campaigns
Photo of Fei Wang

Fei Wang

CarGurus

Fei Wang is a senior data scientist and statistician at CarGurus. His work primarily involves experimental design and causal inference modeling for online and TV advertising. Fei’s research includes statistical machine learning, matrix factorization, optimization, and high-dimensional data modeling. Fei holds a PhD in biostatistics from the University of Michigan.

    Contact us

    confreg@oreilly.com

    For conference registration information and customer service

    partners@oreilly.com

    For more information on community discounts and trade opportunities with O’Reilly conferences

    strataconf@oreilly.com

    For information on exhibiting or sponsoring a conference

    pr@oreilly.com

    For media/analyst press inquires