Presented By O'Reilly and Cloudera
Make Data Work
September 26–27, 2016: Training
September 27–29, 2016: Tutorials & Conference
New York, NY

Elastic data services on Mesos via Mesosphere’s DC/OS

Adam Bordelon (Mesosphere), Mohit Soni (Mesosphere)
2:05pm–2:45pm Wednesday, 09/28/2016
Hadoop internals & development
Location: 3D 10 Level: Intermediate
Average rating: ****.
(4.33, 3 ratings)

Prerequisite knowledge

  • A basic understanding of Hadoop
  • What you'll learn

  • Learn how to use Apache Myriad to run Hadoop on the same machines as the rest of your stack
  • Description

    Most data centers (and many cloud deployments) are statically partitioned into siloed clusters dedicated to running individual datacenter-scale applications, including web services, databases, and batch/stream processing. This static partitioning model limits overall cluster utilization, decreases flexibility, and poses operational challenges. There is an increasing need to integrate big data applications like Apache Hadoop and Apache Spark with other data center services like Apache Cassandra or Apache Kafka, ideally colocating the data with the services that need it.

    Adam Bordelon and Mohit Soni demonstrate how projects like Apache Myriad (incubating) can install Hadoop on Mesosphere DC/OS alongside other data center-scale applications, enabling efficient resource sharing and isolation across a variety of distributed applications while sharing the same cluster resources and hence breaking silos. The multitenancy strategy improves overall cluster utilization and operational efficiency while allowing cluster operators to run multiple isolated data services on the same hardware.

    Adam Bordelon

    Mesosphere

    Adam Bordelon is a distributed systems engineer at Mesosphere and an Apache Mesos committer. Before joining Mesosphere, Adam was lead developer on the Hadoop core team at MapR Technologies, developed distributed systems for personalized recommendations at Amazon, and rearchitected the LabVIEW compiler at National Instruments. He holds a master’s degree from Rice University, where he built a tool to analyze supercomputer performance data for bottlenecks and anomalies.

    Photo of Mohit Soni

    Mohit Soni

    Mesosphere

    Mohit Soni is a distributed applications engineer at Mesosphere building the Datacenter Operating System (DCOS). Previously Mohit worked as an engineer on the Platform team at eBay, where he focused on maximizing efficiency, increasing agility, and reducing cost. Mohit has presented at DockerCon 2014, Hadoop India Summit 2011, and BarCamp 2010. You can follow him as mohitsoni on GitHub and Twitter.