Skip to main content

Mesos: Elastically Scalable Operations, Simplified

Adam Bordelon (Mesosphere), Niklas Nielsen (Mesosphere, Inc.)
Average rating: *****
(5.00, 1 rating)

Apache Mesos is a cluster manager that provides efficient resource isolation and sharing across distributed applications. It can run and manage Apache Hadoop, Apache Spark, MPI, Hypertable, Storm, Chronos, Marathon, and other applications on a dynamically shared pool of nodes. Mesos provides distributed systems primitives that make it easy to build scalable, fault-tolerant frameworks.

The biggest user of Mesos is currently Twitter, where it runs on tens of thousands of cores. Airbnb runs all of their data infrastructure on it, processing petabytes of data. At such large scale, it becomes increasingly important to provide developers with direct access to cluster resources, for scaling and introducing new services. In this way, Mesos speeds development and makes life easier for the data center operator.

This talk will cover:

  • Mesos as a cluster resource manager.
  • Mesos as a distributed systems framework and library.
  • Main takeaways from large-scale adoption.
  • Our vision for future developments in Mesos data center schedulers.

Adam Bordelon


Adam is a distributed systems engineer at Mesosphere and works on Apache Mesos.
Before joining Mesosphere, Adam was lead developer on the Hadoop core team at MapR Technologies, he developed distributed systems for personalized recommendations at Amazon, and he rearchitected the LabVIEW compiler at National Instruments. He completed his Master’s degree at Rice University, building a tool to analyze supercomputer performance data for bottlenecks and anomalies.

Photo of Niklas Nielsen

Niklas Nielsen

Mesosphere, Inc.

Niklas is also a distributed systems engineer at Mesosphere and a committer in the Apache Mesos Open Source project. Before joining Mesosphere, Niklas worked at Adobe as a virtual machine and compiler engineer and completed his Master’s degree doing supercomputer debugger design at Lawrence Livermore National Laboratory.