Presented By
O’Reilly + Cloudera
Make Data Work
29 April–2 May 2019
London, UK

The changing face of ETL: Event-driven architectures for data engineers

Robin Moffatt (Confluent)
12:0512:45 Wednesday, 1 May 2019
Data Engineering and Architecture
Location: Capital Suite 8/9
Average rating: ****.
(4.21, 14 ratings)

Who is this presentation for?

  • Data engineers, architects, and developers

Level

Intermediate

What you'll learn

  • Understand the power of events and unbounded data and why streaming is for everyone
  • Learn where a streaming platform fits in an analytic architecture and how event-driven architectures can enable greater scalability and flexibility of systems both now and in the future

Description

Data integration in architectures built on static, update-in-place data stores inevitably end up with pathologically high degrees of coupling and poor scalability. This has been the standard practice for decades as we attempt to build data pipelines on top of databases that do a poor job modeling the fundamental objects that drive our businesses and systems: events.

Events carry both notification and state and form a powerful primitive on which to build systems for developers and data engineers alike. Developers benefit from the asynchronous communication that events enable between services, and data engineers benefit from the integration capabilities. Everyone gains from using the scalable and resilient standards-based streaming platform.

Robin Moffatt discusses the concepts of events, their relevance to software and data engineers, and their ability to unify architectures in a powerful way. Join in to learn why analytics, data integration, and ETL fit naturally into a streaming world. Along the way, Robin will lead a hands-on demonstration of these concepts in practice and commentary on the design choices made.

Topics include:

  • The power of events and unbounded data
  • Streaming is not just for real-time applications; it’s for everyone
  • Where a streaming platform fits in an analytic architecture
  • How event-driven architectures can enable greater scalability and flexibility of systems both now and in the future
Photo of Robin Moffatt

Robin Moffatt

Confluent

Robin Moffatt is a senior developer advocate at Confluent, the company founded by the original creators of Apache Kafka, as well as an Oracle Groundbreaker Ambassador and ACE Director Alumnus. His career has always involved data, from the old worlds of COBOL and DB2 through the worlds of Oracle and Hadoop and into the current world with Kafka. His particular interests are analytics, systems architecture, performance testing, and optimization. He blogs at http://cnfl.io/rmoff and http://rmoff.net/ (and previously http://ritt.md/rmoff) and can be found tweeting grumpy geek thoughts as @rmoff. Outside of work, he enjoys drinking good beer and eating fried breakfasts, although generally not at the same time.