What's New in Hadoop MapReduce 2?

Hadoop: Tools & Technology
Location: Room 1-6 Level: Intermediate
Average rating: ****.
(4.67, 3 ratings)

Apache Hadoop 2 includes a new MapReduce engine, which has a number of advantages over the previous implementation, including better scalability and resource utilization. The new implementation is built on a general resource management system for running distributed applications called YARN. In this I talk Tom will walk through the new architecture and compare and contrast MapReduce 2 running on YARN with the architecture of MapReduce 1. He will also go through what you need to do to run your MapReduce applications on the new implementation.

Another interesting feature of YARN is that it is more general than MapReduce, and already a number of parallel processing applications have appeared that run on YARN, such as MPI. The talk will briefly discuss some of these applications and how to go about writing your own.

Photo of Tom White

Tom White


Tom White is one of the foremost experts on Hadoop. He has been an Apache Hadoop committer since February 2007, and is a member of the Apache Software Foundation. His book Hadoop: The Definitive Guide (O’Reilly) is recognized as the leading reference on the subject. In 2011, Whirr, the project he founded to run Hadoop and other distributed systems in the cloud, became a top-level Apache project.

Tom is a software engineer at Cloudera, where he has worked since its foundation, on the core distributions from Cloudera and Apache. Previously he was an independent Hadoop consultant, working with companies to set up, use, and extend Hadoop. He has written numerous articles for O’Reilly, java.net and IBM’s developerWorks, and has spoken at several conferences, most recently at ApacheCon and OSCON in 2011. Tom has a Bachelor’s degree in Mathematics from the University of Cambridge and a Master’s in Philosophy of Science from the University of Leeds, UK.


Sponsorship Opportunities

For information on exhibition and sponsorship opportunities, contact Susan Stewart at sstewart@oreilly.com or +1 (707) 827-7148

Media Partner Opportunities

For information on trade opportunities contact Kathy Yu at mediapartners

Press and Media

For media-related inquiries, contact Maureen Jennings at maureen@oreilly.com

Contact Us

View a complete list of Strata contacts.