Engineering the Future of Software
29–31 Oct 2018: Tutorials & Conference
31 Oct–1 Nov 2018: Training
London, UK

Event Streaming as a Source of Truth

Benjamin Stopford (Independent)

Who is this presentation for?

Architect

Prerequisite knowledge

Familiarity with Messaging, Microservices, Databases and basic system architecture.

What you'll learn

- How the Event Streaming as a Source of Truth pattern provides a powerful alternative to using RPCs, Enterprise Messaging or a Shared Database to communicate and share data across different applications or microservices. - How Event Sourcing, Event Driven Architectures and Stream Processing relate to this pattern. - How the event driven mind-set alters the way we think about system design. - How stream processing tools help us manage event streams directly, as they move around the organisation.

Description

One of the most interesting, and provocative patterns to face the software architecture community is the use of event streaming as a source of truth. A pattern where replayable logs, like Apache Kafka, are used for both communication as well as event storage, incorporating the retentive properties of a database in a system designed to share data across many teams, clouds and geographies.

This is a concept Ben Stopford believes to be transformative. Such a bold claim should of course be met with a healthy degree of scepticism, but the interesting thing about communication patterns is that their value comes from often subtle, systemic effects, particularly where humans are involved. You will be familiar with these already: Email, Twitter, Slack and Facebook are all conceptually similar forms of communication, but display very different dynamics in practice, yet zeroing in on exactly why these tools operate and feel so different is rarely as simple as it may seem.

So, in this talk Ben explores not only the Event Streaming pattern, but also the systemic effects it has on the architectures we build around it, digging into where the value really lies. He examines the relationship between events, event sourcing, and stream processing, leading the audience to the idea of a database unbundled, or turned inside out. He also explores how the pattern encourages subtler systemic effects: easier evolution, a more ephemeral view on data, and systems that seamlessly span departments, cloud providers and geographies.

Photo of Benjamin Stopford

Benjamin Stopford

Independent

Ben Stopford is a technologist working in the Office of the CTO at Confluent, Inc. (the company behind Apache Kafka), where he has worked on a wide range of projects, from implementing the latest version of Kafka’s replication protocol through to developing strategies for streaming applications. Before Confluent, Ben led the design and build of a company-wide data platform for a large financial institution, as well as working on a number of early service-oriented systems, both in finance and at ThoughtWorks. He is the author of the book “Designing Event Driven Systems”, O’Reilly, 2018. Find out more at http://benstopford.com.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)