Engineering the Future of Software
Feb 3–4, 2019: Training
Feb 4–6, 2019: Tutorials & Conference
New York, NY

The next data engineering architecture: Beyond the lake (sponsored by Thoughtworks)

Zhamak Dehghani (ThoughtWorks)
2:15pm–3:05pm Wednesday, February 6, 2019
Sponsored
Location: Sutton North
Average rating: ****.
(4.67, 6 ratings)

Prerequisite knowledge

  • Familiarity with enterprise and large-scale data engineering platform architecture

What you'll learn

  • Understand the challenges of current data engineering architecture and organizational structure
  • Learn a new approach to building data engineering architecture and organizational structure

Description

The data engineering domain has benefited from many advances over the recent years, including streaming technologies, on-demand infrastructure, and the proliferation of cloud-native tooling to digest, process, store, and serve data at scale. Despite these advances, one thing remains underdeveloped: the approach to the overall data engineering platform architecture and team structure. The current state of the art, unfortunately, remains centralized, functionally organized and managed by hyperspecialized teams, difficult to scale, and hard to leverage.

Zhamak Dehghani shares observations and lessons learned working with and consulting many organizations that are busy developing their data engineering capabilities and technology. Zhamak then introduces an alternative data engineering architecture, organizational, and operational model that can enable your organization to become truly data driven—a decentralized and ubiquitous data engineering platform that enables any team to produce or consume data at scale with low overhead.

This session is sponsored by Thoughtworks.

Photo of Zhamak Dehghani

Zhamak Dehghani

ThoughtWorks

Zhamak Dehghani is a principal consultant at ThoughtWorks focusing on distributed systems architecture and digital platform strategy for the enterprise. She is a member of the ThoughtWorks Technology Advisory Board and contributes to the creation of the ThoughtWorks Technology Radar. Zhamak has worked as a software engineer and architect for 20 years in the areas of distributed computing communications and embedded device technologies and has contributed to multiple patents on embedded mobile sensing devices.