This talk introduces the audience to Apache Bigtop – a project aimed at developing packaging and tests within the Hadoop ecosystem. By making use of various packages available through Apache Bigtop, we would learn how to set up a cluster with Hadoop, Hive and HBase installed and configured in under 15 minutes. Subsequently, we will run some example MapReduce jobs, Hive and HBase queries to validate the setup.
Apache Bigtop is a project aimed at development of packaging and tests within the Hadoop ecosystem. Bigtop packages various Big Data related open source projects like Hadoop, Hive, Hbase, etc. and makes them available as architecture specific deb/rpm packages. These packages can then be easily installed using available Operating System installers like apt-get, zypper or yum. One of the longer term goals of Apache Bigtop is to serve as a reference to eventually get Hadoop introduced into most Linux distributions.
Time permitting, we will introduce another important function of Apache Bigtop – interoperability testing. Given the dependencies between various projects and their sheer number of versions, it is a daunting task test the interoperability of various components. One of goals of Bigtop is to address this problem and we will learn how Bigtop does so.
The talk will end with a short Q/A session.
Mark is a software engineer working on Apache Spark at Cloudera. He is a co-author of Hadoop Application Architectures and also wrote a section in Programming Hive book.
Mark is also a committer on Apache Bigtop and a committer and PMC member on Apache Sentry. He has contributed to a number of open source projects including Apache Hadoop, Apache Hive, Apache Sqoop and Apache Flume projects.
Mark is sought after speaker on topics related to Big Data at various national and international conference. He occasionally blogs on topics related to technology on his blog.
Comments on this page are now closed.
For information on exhibition and sponsorship opportunities at the conference, contact Sharon Cordesse at (707) 827-7065 or firstname.lastname@example.org.
View a complete list of OSCON contacts