Big Data, Lean Startup: Data Science on a Shoestring

Practitioner
Location: Mission City M
Average rating: ***..
(3.93, 29 ratings)

As Hal Varian famously predicted back in 2009, data scientists have seen explosive job-demand in recent years. Even in the face of a global recession, in the last two years, the average salary for data scientists and Hadoop engineers is spiking well over $100k all around the country.

So, how do you build a crack team of data scientists and engineers on a shoestring budget? At Infochimps, we’ve been able to craft a team of data scientists by drawing upon smart, enthusiastic hires from our nearby university in untraditional areas such as non-linear dynamics and statistical physics, and equipping them with the process and tools that accelerate their transformation into bona fide data scientists. Drawing from both his experiences as a teacher and his vast programming and data experience, Flip developed our methodology that embraces failure and constantly pushes people outside of their comfort zone, ultimately resulting in better, smarter scientists.

Our unique approach to data science is well suited for our small, nimble organization. Emphasizing program simplicity and programmer fun over runtime efficiency lets us run the minimum viable experiments that a lean startup requires. Cloud computing and dev ops let us achieve massive horizontally scaling with minimal distraction.

In this 40-minute presentation from the co-founder of Infochimps, Flip Kromer will walk you through our process of recruitment and data science methodology within a fast-paced and pivotal startup. In the end, you’ll learn how to make data scientists out of just about any (smart, driven) simian.

Photo of Philip (Flip) Kromer

Philip (Flip) Kromer

CSC

Flip is the Founder and CTO of Infochimps, a data marketplace to find any data-set in the world. He holds a B.S. in Physics and Computer Science from Cornell University and attended graduate school in Physics at the University of Texas at Austin. Flip enjoys riding his bicycle around Austin, eating homemade soup, and playing extreme Scrabble. The idea for Infochimps was inspired by Flip’s abhorrence of redundancy and desire to make the world a happier place for data lovers of all kinds.

Comments on this page are now closed.

Comments

Stephen O'Grady
02/04/2011 5:00am PST

One of the better talks I attended at Strata, which is saying something. In particular, the aggressive pragmatism of the presentation was very rewarding in that it encourages start ups to think hard about the basic decisions they make, and actually do the math on them.

Highly recommended.

Sponsors

  • Thomson Reuters
  • EMC Data Computing Division
  • EnterpriseDB
  • Microsoft
  • Gnip
  • Rackspace Hosting
  • IBM
  • Windows Azure MarketPlace DataMarket
  • Amazon Mechanical Turk
  • Amazon Web Services
  • Aster Data
  • Cloudera
  • Clustrix
  • DataStax, Inc. (formerly Riptano, Inc.)
  • Digital Reasoning Systems
  • Heritage Provider Network
  • Impetus
  • Jaspersoft
  • Karmasphere
  • LinkedIn
  • MarkLogic
  • Pentaho
  • Pervasive
  • Revolution Analytics
  • Splunk
  • Urban Mapping
  • Wolfram|Alpha
  • Esri
  • ParAccel
  • Tableau Software

Sponsorship Opportunities

For information on exhibition and sponsorship opportunities at the conference, contact Susan Young at syoung@oreilly.com

Download the Strata Sponsor/Exhibitor Prospectus

Contact Us

View a complete list of Strata Contacts