4–7 Nov 2019

Think local: Reducing coordination and improving performance in your distributed systems

John Mumm (Wallaroo Labs)
15:0015:45 Wednesday, 6 November 2019
Location: M6/M7
Secondary topics:  Overview

Who is this presentation for?

  • Software engineers and software architects

Level

Beginner

Description

You build distributed systems to maintain availability and improve performance. However, as soon as distributed state is thrown into the mix, you have to start thinking about the performance costs of accessing that state and keeping it current. Many of the worst problems concern the need for coordination across nodes and across systems within a node. Drawing on the lessons he’s learned working on Wallaroo, a high-performance, open source framework for building stateful distributed applications, John Mumm outlines strategies for avoiding coordination and relying on local knowledge wherever possible.

John outlines when you might need to rely on coordination in a distributed system. Coordination has its advantages, but it also normally entails costs regarding availability, throughput, latency, and scalability. Seen from one perspective, the question of whether you need to coordinate boils down the question of whether you need to share global knowledge. For this reason, John looks more closely at what knowledge means in a distributed system and how, given different system requirements, you can make a choice regarding how much you should rely purely on local knowledge, allowing you to avoid coordination.

You’ll examine the ways you might think local in your own distributed systems, drawing on John’s experience building Wallaroo. In particular, you’ll look at using actors as your units of concurrency, avoiding the kinds of coordination that arise from sources ranging from locks to stop-the-world garbage collection; relying on state entities as transaction boundaries, thereby avoiding the coordination involved in distributed transactions; and relying on in-memory application state in order to avoid coordination with external data stores. Each of these strategies involves trade-offs, so you’ll need to think carefully about your own particular use cases and requirements.

Prerequisite knowledge

  • A basic understanding of the principles underlying distributed systems
  • Experience with software engineering

What you'll learn

  • Learn to rethink distributed system architecture in order to avoid coordination and global knowledge wherever possible
Photo of John Mumm

John Mumm

Wallaroo Labs

John Mumm is a software architect at Wallaroo Labs based in Netherlands, where he works on an open source high-performance framework for building stateful distributed applications. John holds a PhD in philosophy.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)

Contact us

confreg@oreilly.com

For conference registration information and customer service

partners@oreilly.com

For more information on community discounts and trade opportunities with O’Reilly conferences

Become a sponsor

For information on exhibiting or sponsoring a conference

pr@oreilly.com

For media/analyst press inquires