Adventures in accidental human architecture

Description

In the 1770s, Wolfgang von Kempelen built an amazing machine: the Mechanical Turk, an automaton that played chess and played it well. He and his chess-playing automaton toured the world and defeated heads of state. However, it was an elaborate hoax. The machine cleverly concealed a human, who actually operated the machine.

As software architects, we build complex systems, and we draw boxes and arrows to capture the structure that supports the system. We have high expectations for the performance, scalability, and quality of the software that the boxes represent. But sometimes, accidentally, we build humans into those boxes, and human intervention becomes essential to keeping the system running. These software equivalents to Mechanical Turks reveal themselves in production when engineers find themselves constantly hand-tuning the system or having to initiate common workflows through support tickets. The fragility of such a system becomes obvious when the it fails as soon as the human operators step away.

Michelle Brush explores the architectural and organizational anti-patterns that lead to systems that require constant human intervention, discusses heuristics for discovering the humans in the machines in existing systems, and shares guiding principles and practices for removing the people from systems through automation, resiliency, observability, and a little usability.