Docker makes it easy to bundle an application with its dependencies and provide full isolation, and YARN now supports Docker as an execution engine for submitted applications. Daniel Templeton explains how YARN’s Docker support works, why you’d want to use it, and when you shouldn’t.
Daniel starts with a very short summary of Docker and the use cases that drive its adoption before jumping into how Docker is supported on YARN. Daniel covers container executors and the various options and digs into the Docker support provided by the LinuxContainerExecutor, and he’ll talk about what it doesn’t do yet, where that creates problems, and the work going on in the community to address the gaps. Along the way, Daniel also explores Apache Slider and the Docker capabilities it brings as it’s being merged into YARN and the use cases for when you’d want to use Docker in a Hadoop environment and when there might be a better alternative, covering both traditional applications, like MapReduce jobs, and longer-running services.
You’ll leave with a concrete idea of what’s possible today, what’s coming soon, and when to look to Docker as the solution.
Daniel Templeton has a long history in high-performance computing, open source communities, and technology evangelism. Today Daniel works on the YARN development team at Cloudera, focused on the resource manager, fair scheduler, and Docker support.
©2017, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • email@example.com
Apache Hadoop, Hadoop, Apache Spark, Spark, and Apache are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries, and are used with permission. The Apache Software Foundation has no affiliation with and does not endorse, or review the materials provided at this event, which is managed by O'Reilly Media and/or Cloudera.