Presented By O'Reilly and Cloudera
Make Data Work
Sept 29–Oct 1, 2015 • New York, NY

Death of the click: How big data is killing your favorite metrics

Claudia Perlich (Dstillery)
1:15pm–1:55pm Wednesday, 09/30/2015
Data-driven Business
Location: 1 E10 / 1 E11 Level: Intermediate
Tags: media
Average rating: ****.
(4.15, 20 ratings)
Slides:   1-PDF 

Description

Data driving decisioning is not an invention of the recent past. We have a waste collection of metrics, KPIs, and other statistics that help decision makers to be more data driven. While the recent technological advances have increased our ability to see the world through the objective lenses of data, one unintended side effect of exactly this technology is the diminishing value of many metrics we cherish.

Consider the clickthrough rate (CTR) in advertising. With the rise of Doubleclick, CTR has been a core metric of an advertiser’s ability to identify a relevant audience: more clicks equals more interest equals more qualified potential consumers. While this might have been true in the past, the sad reality is that measures like CTR have lost their meaning.

The issue is not as much adversarial attempts of gaming, but the fact that even highly correlated proxies like CTR lose their power once optimization can draw from more granular information. Given much more detailed information, modern optimization approaches can find signals in the noise, and identify users with poor vision or bad fine motor skills who accidentally (but not randomly) click on ads, rather than interested potential consumers. This fundamental issue extends beyond clicks and calls for a close examination of many of our favorite KPIs.

Photo of Claudia Perlich

Claudia Perlich

Dstillery

Prior to joining Dstillery (former Media6Degrees), Claudia Perlich spent five years working at the Data Analytics Research group at the IBM T.J. Watson Research Center, concentrating on research in data analytics and machine learning for complex real-world domains and applications. She has been published in over 30 scientific publications and holds multiple patents in the area of machine learning. Claudia has won many data mining competitions, including the prestigious 2007 KDD CUP on movie ratings, the 2008 KDD CUP on breast-cancer detection, and the 2009 KDD CUP on churn and propensity predictions for telecommunication customers. Claudia received her Ph.D. in Information Systems from Stern School of Business, New York University in 2005, and holds a Master of Computer Science from Colorado University.