Presented By O'Reilly and Cloudera
Make Data Work
Sept 29–Oct 1, 2015 • New York, NY

The business case for Spark, Kafka, and friends

Edd Wilder-James (Google)
9:55am–10:30am Tuesday, 09/29/2015
Data 101
Location: 3D 05 / 08 Level: Non-technical
Average rating: ***..
(3.88, 17 ratings)
Slides:   1-PDF 

Spark is white-hot, but why does it matter? Some technologies cause more excitement than others, and at first the only people who understand why are the developers who use them. This talk provides a tour through the hottest emerging data technologies of 2015 and explains why they’re exciting, in the context of the new capabilities and economies they bring.

The secret power of big data technologies is that they promote flexible development patterns, economic scaling, and are ready to adapt to business needs—but years of focusing on the label “big” has obscured much of the value to those approaching the topic. Skepticism and hype-fatigue are understandable reactions.

We’ll look at the excitement surrounding this year’s emerging platforms of choice, explain where they fit into a complete data architecture, and what they have to offer in terms of new capabilities, efficiencies, and economies of use.

Among other topics, we’ll cover:

  • Spark
  • Kafka
  • Docker and containers
  • The state of Hadoop
  • Big data in the cloud

Note: This list will be updated as the summer wears on and key technologies rise to importance.

This talk is intended for business leaders, and anybody who wants to articulate the value of technology to the business.

Photo of Edd Wilder-James

Edd Wilder-James


Edd Wilder-James is a strategist at Google, where he is helping build a strong and vital open source community around TensorFlow. A technology analyst, writer, and entrepreneur based in California, Edd previously helped transform businesses with data as vice president of strategy for Silicon Valley Data Science. Formerly Edd Dumbill, Edd was the founding program chair for the O’Reilly Strata Data Conference and chaired the Open Source Software Conference for six years. He was also the founding editor of the peer-reviewed journal Big Data. A startup veteran, Edd was the founder and creator of the Expectnation conference management system and a cofounder of the Pharmalicensing online intellectual property exchange. An advocate and contributor to open source software, Edd has contributed to various projects such as Debian and GNOME and created the DOAP vocabulary for describing software projects. Edd has written four books, including Learning Rails (O’Reilly).