Apache Hadoop has become popular from its specialization in the execution of MapReduce programs. However, it has been hard to leverage existing Hadoop infrastructure for various other processing paradigms such as real-time streaming, graph processing and message-passing. That was true until the introduction of Apache Hadoop YARN in Apache Hadoop 2.0. YARN supports running arbitrary processing paradigms on the same Hadoop cluster. This allows for development of newer frameworks as well as more efficient implementations of existing frameworks that can all run on and share the resources of a single multi-tenant YARN cluster. This talk will give a brief introduction to YARN. We will illustrate how to create, test and deploy applications on YARN. We will show examples of different applications such as Apache Tez and Apache Giraph that can leverage YARN and present best practices/guidelines on building applications on top of Apache Hadoop YARN.
Siddharth Seth works as a software engineer at Hortonwork, where he works on the Apache Tez project and the Apache Hadoop project – with a focus on YARN and MapReduce. He is a member of the Apache Tez PPMC and the Apache Hadoop PMC. Prior to this he spent several years working on search platforms and Oozie at Yahoo.
Hitesh Shah currently works on various things related to Apache Hadoop at Hortonworks. He is an Apache Hadoop committer as well as a PPMC member of the Apache Ambari and Apache Tez projects. Earlier to that, he spent 8+ years at Yahoo! building various frameworks all the way from data storage platforms for social content to a multi-threaded event-driven framework for building high throughput advertising serving platforms.
Comments on this page are now closed.
For exhibition and sponsorship opportunities, contact Susan Stewart at email@example.com
For information on trade opportunities with O'Reilly conferences email mediapartners
For media-related inquiries, contact Maureen Jennings at firstname.lastname@example.org
View a complete list of Strata + Hadoop World 2013 contacts