Qubole started out by offering Hadoop as a service in AWS. Over time, it extended its big data capabilities beyond Hadoop and its cloud infrastructure support beyond AWS. To do this, Qubole needed to build a simple, cloud-agnostic, multipurpose provisioning tool that could be extended for further engines and further cloud support. Sriram Ganesan and Prakhar Jain describe how and why Qubole built cluster management tool Cloudman to deploy Spark, Hadoop, and other big data engines across several public IaaS cloud platforms, such as AWS, Microsoft Azure, and Oracle Public Cloud.
Sriram Ganesan is a member of the technical staff at Qubole, where he works on HBase and cluster orchestration. Previously, Sriram was at Directi, where he worked on scaling the backend of leading chat app Talk.to. Sriram holds a bachelor of computer science engineering from the National Institute of Technology, Trichy, India.
Prakhar Jain is a member of the technical staff at Qubole, where he works on the cluster orchestration stack. Prakhar holds a bachelor of computer science engineering from the Indian Institute of Technology, Bombay, India.
©2017, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • firstname.lastname@example.org
Apache Hadoop, Hadoop, Apache Spark, Spark, and Apache are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries, and are used with permission. The Apache Software Foundation has no affiliation with and does not endorse, or review the materials provided at this event, which is managed by O'Reilly Media and/or Cloudera.