Engineering the Future of Software
Feb 25–26, 2018: Training
Feb 26–28, 2018: Tutorials & Conference
New York, NY

Event storage in a distributed system

Stephen Pember (ThirdChannel)
10:45am–12:15pm Wednesday, February 28, 2018
Secondary topics:  Best Practice, Overview
Average rating: *****
(5.00, 1 rating)

Who is this presentation for?

  • Architects, tech leaders, and senior developers

Prerequisite knowledge

  • Familiarity with distributed systems, reactive programming, DDD, and CQRS

What you'll learn

  • Understand the value of event storage in distributed systems
  • Learn how to handle, store, and read these events

Description

Developers building modern distributed systems are embracing techniques that rely on asynchronous event handling for service communication. Reactive systems demand it. While it’s possible to ignore these events once they’ve been acted upon, in the long run they have massive analytical value. A wise developer would persist every event that flows through their system by building an event store mechanism.

In this context an event store isn’t one framework, tool, or library. It’s a pattern or a mechanism designed to provide queryable access to your events in some fashion. Event storage offers many practical benefits to distributed systems providing complete state changes over time. This allows for interesting analytical insights (provided you can mine these events appropriately), easy bootstrapping of new services into the system, and the possibility of viewing the state of the system at any point in time, among other things. However, there are a number of challenges when building an event store mechanism: Where should these events physically persist? Do we create one centralized event store, or do we rely on each service to store their own events? And how do we go back in time and query for events?

Stephen Pember explores some of the problems you may encounter and shares real-world patterns for working with event storage, championing the use of asynchronous events as a primary means of communication and techniques for persisting and accessing those events. Stephen briefly explores reactive programming and CQRS in order to see how they result in efficient, decoupled services before demonstrating how event storage can be a powerful analytical tool that enables custom projections of your data. Stephen weighs the merits of a centralized event store against local event stores and explores practical code examples for event design and event handling flow, based on his real-world experience. Although you’ll hear the word “event” so many times you’ll probably be sick of hearing it, you’ll walk away ready to build an event storage mechanism in your own system.

Photo of Stephen Pember

Stephen Pember

ThirdChannel

Stephen Pember is the CTO of ThirdChannel, a startup in Boston, MA, that crowdsources data from thousands of people through their smartphones and then presents that data through visualization and analytic tools in order to improve the operations of large product companies. In this role, Steve is designing and building a reactive, event-driven platform with a frontend based around Backbone.js. Steve was formally a principal consultant with Cantina, a technology agency specializing in utilizing the forefront in web technologies to construct top-notch experiences. His passion lies in architecting and developing performant, scalable, full stack systems for the web.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)