In the 2.4.0 release, a new and improved HTML5-based native Web UI was introduced in Hadoop. The new WebUI used the WebHDFS API to talk to HDFS. However, it lacked write capabilities. Many customers requested this slick new UI to be improved to a full-service interface, including:
1. Ability to create directories: mkdir
2. Ability to change permissions: chmod
3. Ability to change owner and group: chown, chgrp
4. Ability to upload files
5. Ability to delete files and directories
6. Sort based on columns
This presentation will cover the challenges to achieving these features and how these challenges were solved, such as how the WebHDFS API was modified. If there is time, we will also cover how HttpFS was used to act as a single gateway for channeling all traffic.
Although there are several frameworks that also provide this functionality, we believe that such basic features should be included natively in Hadoop, without requiring additional components. We also show how this may be more efficient than an additional server proxying all traffic.
Future upgrades will also be covered, such as functionality for multiple file/directory uploads and other features to further improve the experience of HDFS users.
Ravi Prakash is a Hadoop committer and a senior software engineer at Altiscale. Previously, he was a senior software developer at Yahoo!, where he worked on Hadoop Core development (HDFS, MapReduce, and YARN). Ravi has also worked in software development at Tavare Research Labs and Motorola. Ravi has a BS in computer science from GGS Indraprastha University and an MS in computer science from the University of Southern California.
©2015, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • email@example.com
Apache Hadoop, Hadoop, Apache Spark, Spark, and Apache are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries, and are used with permission. The Apache Software Foundation has no affiliation with and does not endorse, or review the materials provided at this event, which is managed by O'Reilly Media and/or Cloudera.