Presented By O'Reilly and Cloudera
Make Data Work
Feb 17–20, 2015 • San Jose, CA

From MapReduce to Programming Frameworks: Making Sense of Cloud Dataflow, Spark and New Tools for Big Data

Eric Schmidt (Google)
1:30pm–2:10pm Thursday, 02/19/2015
Hadoop & Beyond
Location: 230 C
Average rating: ***..
(3.71, 7 ratings)

Map Reduce, Millwheel and a host of other technologies changed the way programmers approached data problems. Rather than being constrained by the size of the data set, developers could focus on defining the key bits of intelligence they were seeking to find from an almost infinite data pool. While this opened up all kinds of new analysis, it also created a new set of problems — most notably dealing with the complexity of stringing together map reduces and creating end to end programming logic from multiple steps. New technologies like Spark and Cloud Dataflow seek to solve this problem and make Big Data into a concrete set of executable operations. But what makes sense for you? How do they differ and how do they evolve? This talk seeks to make sense of Big Data programming options and give developers a sense as to what is coming next.

Photo of Eric Schmidt

Eric Schmidt


Eric Schmidt is the product management lead for Cloud Dataflow on the Cloud engineering team at Google, where his primary role is to help shape the future of fully managed, large-scale data processing. Eric spends the majority of his time working with existing cloud customers and on-premises developers who are moving their MapReduce and related data processing workloads to the cloud. He led the announcement of Cloud Dataflow (as Google I/O’s 2014 keynote) with the development of a real-time sentiment analysis and results prediction framework for the 2014 World Cup. Eric has a deep passion for user interaction modeling, data modeling, and analytical processing of user behaviors. He also has development experience with .NET, C, JavaScript, Python, and Java and cloud expertise with Microsoft Azure and Amazon Web Services. Previously, Eric was a senior director within Technical Evangelism and Development at Microsoft, where he led a team focusing on the adoption of Microsoft’s devices and cloud services. His team delivered Emmy Award-winning applications for NBC Sports, the Democratic National Convention, NCAA March Madness on Demand, and Major League Soccer. Eric also spearheaded the development of a real-time quality and user behavior system for the online streaming of the 2008, 2010, and 2012 Olympic games.