Skip to main content

The Sidekick Pattern: Using Small Data to Increase the Value of Big Data

Abe Gong (Superconductive Health)
Data Science
Ballroom AB
Average rating: ****.
(4.00, 9 ratings)
Slides:   1-PDF 

This session introduces the Sidekick Pattern: using small amounts of carefully curated data to increase the value of big, messy data sets. Common sources for data sidekicks include surveys, small experiments, crowdsourcing, and public APIs. In the first year of data science at Jawbone, we have used the Sidekick Pattern many times to augment data streams from the UP fitness tracker: movement, sleep, workouts, and food.

With the right tools, data sidekicks are easy to create, and they can accelerate analysis, solve cold start problems, and simplify complicated data pipelines. As a result, data sidekicks are a valuable technique for enabling rapid prototyping, securing organizational buy-in, and bringing new data products to market faster.

The session is geared towards practical answers to the following questions:

  • What makes a good data sidekick?
  • What are the advantages and disadvantages of using data sidekicks?
  • How can I recognize opportunities to use the sidekick pattern?
  • What free/low-cost tools can accelerate the sidekick pattern?
  • What are common mistakes when applying the sidekick pattern?
Photo of Abe Gong

Abe Gong

Founder and CEO, Superconductive Health

Abe Gong is a data scientist at Jawbone, where he builds data products to support the UP fitness tracker.