Building a machine learning framework to measure TV advertising attribution

Description

In the online automotive shopping space, one of the most important, yet undoubtedly complex measurements, is lead attribution. Companies investing in their brand marketing via digital performance marketing, TV advertising, content marketing, etc., need to measure the cost effectiveness of spend for each medium and each campaign. However, advertisers face a challenge in measuring TV’s attribution using nonrandomized and nonexperimental data when digital performance marketing and digital content advertising are running at the same time.

In the case of CarGurus, traffic acquisition had been driven solely by a proprietary algorithmic digital performance marketing model. In June 2017, CarGurus launched its first national television advertising campaign in the US. The data science team was tasked with building a new TV attribution model using a causal inference framework to measure uniquely TV-driven leads and calculate how effective the campaign was—the CPA metric.

Fei Wang takes a deep dive into a novel machine learning tool based on the state space model to quantify casual effects of TV, digital performance marketing, and digital content advertising simultaneously, emphasizing the conceptual framework to explicitly include potential and important causal factors driving leads and interpretability of the causal inference model.

The approach is two-fold. First, by using historical lead data during the pretest period, CarGurus built a state space model to understand the data-generation process before TV advertising. This model then predicts the counterfactual leads during the posttest period. The difference between actual and predicted counterfactual leads is treated as the incremental leads not from digital performance marketing (non-DPM). This non-DPM contribution not only contains the causal effect from the TV spend but may also contain contributions from other factors, for example, improved mobile website design and increased vehicle listings on the CarGurus website during the posttest period. Second, causal effect from the TV ads is obtained by stripping out contributions from factors except TV ads using certain domain knowledge.

You’ll leave having taken a deep dive into the causal inference model, including the state space model with exogenous variables to account for the inherent lack of control groups and lack of randomization with TV advertising campaigns; how to correct for non-TV factors and simultaneously obtain causal effects for TV, digital performance marketing, and digital content advertising; how to measure TV’s lagged effects and some statistical techniques to improve model interpretability; the interplay of TV advertising and digital performance marketing and how to attribute part of your digital performance marketing costs to TV advertising origin; and CPA calculation by identifying extra digital performance marketing cost.