Building and maintaining complex distributed systems
June 19–20, 2017: Training
June 20–22, 2017: Tutorials & Conference
San Jose, CA

How to scale a distributed system

Henry Robinson (Cloudera)
11:25am–12:05pm Thursday, June 22, 2017
Distributed Systems
Location: 230 B
Level: Intermediate
Average rating: *****
(5.00, 3 ratings)

Who is this presentation for?

  • Distributed systems engineers

Prerequisite knowledge

  • A passing familiarity with common distributed systems architectures (client/server, primary and backup, replicated primaries, etc.) (useful but not required)

What you'll learn

  • Learn reusable design ideas for building scalable distributed systems

Description

Despite the continuing high industrial demand for building new distributed systems, there are few institutionalized, commonly applicable techniques and design approaches like those found in other engineering disciplines. Practitioners are left to learn the same lessons over and over again, either through hard-won experience or by stumbling across a relevant paragraph in an academic paper.

Henry Robinson shares practical lessons learned from more than eight years spent building distributed systems using the Hadoop ecosystem (including Apache Zookeeper, Apache Flume, Apache Impala, and more), focusing on the thorny question of how to scale a distributed system. Henry outlines a framework for thinking about the problems of scale (in many dimensions) and effectively navigating the phase transitions between 10-, 100-, and 1,000-node deployments.

Topics include:

  • How much scalability is enough?
  • Modeling system invariants with safety and liveness
  • Managing workloads and knowing when to say no to load
  • The importance of separating control and data
Photo of Henry Robinson

Henry Robinson

Cloudera

Henry Robinson is a software engineer with some experience in open-source distributed systems, including Apache Zookeeper, Apache Flume and Apache Impala.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)