Build Systems that Drive Business
June 11–12, 2018: Training
June 12–14, 2018: Tutorials & Conference
San Jose, CA

Declarative automation for distributed databases

John Miller (Fauna)
4:35pm–5:15pm Wednesday, June 13, 2018
Building Secure Systems
 Location: LL21 E/F Level: Advanced
Secondary topics: Distributed State
Average rating: *....
(1.33, 3 ratings)

Prerequisite knowledge

  • Experience with networked operational environments
  • Familiarity with cluster automation tools or container environments, such as Ansible or Kubernetes
  • A basic understanding of database systems, such as MySQL or Cassandra

What you'll learn

  • Explore patterns for building declarative management tooling for distributed stateful systems

Description

Deploying and operating distributed databases, particularly ones that offer strong data consistency guarantees, is fundamentally more complicated than managing stateless systems—and fraught with risks. Data loss, bad cluster state, operational dead-ends: every DevOps engineer has seen the effects of a poorly designed operations component in their database.

This complexity makes achieving the ideal of declarative automation—where the operator is able to simply tell the system to migrate to a desired state—a nontrivial engineering task. However, there are some general patterns that can be applied to the design of cluster automation systems in order to make their development tractable.

John Miller explores these patterns and walks you through some real-world implementation examples, such as the operational tooling from Twitter’s earlier days of sharded MySQL, and a tour of FaunaDB’s modern declarative cluster management system.

Photo of John Miller

John Miller

Fauna

John Miller III is a senior developer at Fauna since 2017 and focuses on driving advanced technologies into the core server. Prior to Fauna, John spent 28 years as a Senior Technical Staff Member (STSM) at IBM. In his last year, he lead IBM’s Analytics Platform development group and brought advanced technology to the Next Generation Analytic Platform which served the needs of data engineers and data scientists. In addition, he spent 26 years with Informix and 7 years as chief architect where he has designed many components and features, including the Hybrid JSON functionality. John has also focused on the embed space, including a completed redesign of the Informix administration system and the OpenAdmin Tool for Informix. He has led key improvements in database backups, high availability, benchmarking, scale up and scale out architectures, and has patents granted and pending in these areas. John was also a member of the IBM Academy of Technology, on numerous architecture and technical promotion boards, and involved in many cross IBM technology initiatives.