Build resilient systems at scale
28–30 October 2015 • Amsterdam, The Netherlands

Managing secrets at scale

Alex Schoof (Fugue)
13:45–14:25 Friday, 30/10/2015
Location: G106/107
Average rating: ****.
(4.58, 33 ratings)
Slides:   1-PDF 

Prerequisite Knowledge



Modern systems are full of secrets. There are secrets we think about all the time, like private keys for SSL certificates, or the prod database password, and there are secrets that we ignore or forget like the secret used to generate HMACs for session cookies. All these secrets present management hurdles:

  • They need to be safely and securely distributed to servers that need them
  • They must have some kind of access control to let us decide who can and who cannot use them
  • We need some mechanism to revoke and rotate them, either due to a compromise, or just because they’re getting old

As applications move from the laptop into the cloud (or data center), these issues are usually not considered. Too often we just SCP keys around our environments, or bake them into the deployment image. Haphazard management of keys can lead to management headaches in the best cases, and compromise in the worst.

In this session, we will take a step back and look at secrets management as an integral part of your environment. We will talk about what actually needs to be protected, and what we are protecting against. Using and managing secrets means that we need a set of operations that are useful to both applications and operators. We will talk about the lifecycle of secrets, and how building mechanisms to allow for the easy aging-out of keys makes management easier.

These issues will be discussed both at the architectural and the practical level. We’ll look at the core functionality needed by these systems, how to build them, and look at some existing open source systems that help make secrets management easier.

Photo of Alex Schoof

Alex Schoof


As a security-focused software engineer, Alex Schoof has been designing and building systems ranging from GPU-accelerated network analytics to crypto-hardware-as-a-service offerings for large enterprises, startups, and everything in between. He is currently a principal engineer at Fugue, where he works on distributed coordination systems and secure execution environments.