Making Open Work
May 8–9, 2017: Training & Tutorials
May 10–11, 2017: Conference
Austin, TX

Working with cloud filesystems: From semantics to application

Sean Mackrory (Cloudera)
4:15pm4:55pm Thursday, May 11, 2017
Data, Big and Small, Infrastructure
Location: Ballroom F
Level: Intermediate
Average rating: *****
(5.00, 1 rating)

Who is this presentation for?

  • Software engineers and architects

Prerequisite knowledge

  • A working knowledge of Linux
  • A basic understanding of Hadoop (useful but not required)

What you'll learn

  • Understand why moving to the cloud for storage is often a good idea
  • Gain an overview of filesystems in public cloud infrastructures and learn best practices for implementing them

Description

What’s in a filesystem? Whether you’re building a new application in a public cloud such as AWS, Azure, or GCE, porting an existing application to use a cloud file store, or adjusting your code to interact with a cloud-native file store, there is a lot to consider.

Sean Mackrory offers an overview of and best practices for filesystems in public cloud infrastructures as they relate to traditional filesystems. Sean explores some of the considerations with filesystems such as Amazon’s S3, as well as network-attached storage, such as EBS or Microsoft’s SSD-backed premium storage. Many of the examples will relate to Hadoop, namely moving from HDFS to S3. You’ll leave with a map to navigate the waters, based on the testing, tuning, and best practices that ensure that your application and cloud filesystem will work together well.

Photo of Sean Mackrory

Sean Mackrory

Cloudera

Sean Mackrory is a software engineer at Cloudera, where he works on cloud filesystem connectors, in particular s3a for Amazon’s S3 and WASB for Microsft’s Azure platform. Sean has extensive knowledge of Linux packaging and has worked on Cloudera’s build and packaging frameworks.