Build & maintain complex distributed systems
October 1–2, 2017: Training
October 2–4, 2017: Tutorials & Conference
New York, NY

Have you tried turning it off and turning it on again?

Tanya Reilly (Google)
2:25pm3:05pm Wednesday, October 4, 2017
DevOps & Tools, Systems Engineering
Location: Gramercy Level: Intermediate
Average rating: *****
(5.00, 5 ratings)

Who is this presentation for?

  • Systems engineers and architects

Prerequisite knowledge

  • A basic familiarity with distributed systems

What you'll learn

  • Explore disaster recovery best practices

Description

Most of us have a backup strategy, many of us have a restore strategy, and several of us have even fully tested these strategies. But even simple sites may be difficult to recover after a disaster. Tanya Reilly explains why backups are not enough. Complex systems are much harder to reason about and can even be coupled together in ways that make them unrecoverable.

Tanya explores the parts of disaster recovery you might be less prepared for and the dependencies that you might not think about until one day when you really do turn an entire service, entire site, or (perish the thought) an entire company off and on again. You’ll learn why the best laid fallback plans tend to go wrong and why you should start deliberately managing your dependencies long before you think you need to. Along the way, Tanya also covers the dependency cycles that make it difficult or impossible to restart groups of systems—like where do you store the documentation on how to recover the documentation server?

Photo of Tanya Reilly

Tanya Reilly

Google

Tanya Reilly is a system administrator and site reliability engineer at Google, where she works on low-level infrastructure like distributed locking, load balancing, and bootstrapping. Previously, she was a system administrator at Eircom.net, Ireland’s largest ISP, and the entire IT Department for a small software house.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)