Presented By O’Reilly and Cloudera
Make Data Work
21–22 May 2018: Training
22–24 May 2018: Tutorials & Conference
London, UK

You’re doing it wrong: How Zoomdata rearchitected streaming

Erin Recachinas (Zoomdata)
11:1511:55 Thursday, 24 May 2018
Secondary topics:  Visualization, Design, and UX
Average rating: ****.
(4.00, 2 ratings)

Who is this presentation for?

  • Engineers and architects

What you'll learn

  • Understand the pie chart problem and its implications on streaming data
  • Explore Zoomdata's modern streaming architecture

Description

The ability to make quick decisions based on real-time data is important, but most business decisions aren’t made solely on a “window” of the most recent information. Analysts, executives, and decision makers everywhere want historical trends so they can look at what’s happening now in the context of historical data.

Big data application Zoomdata brought streaming and historical data together, addressing the pie chart problem. However, streaming in the age of the IoT requires handling of data in volumes not seen in traditional feeds, and Zoomdata’s legacy streaming architecture wasn’t extensible enough to meet the emerging and future needs of this evolving space.

Erin Recachinas explains how Zoomdata moved to a scalable microservice architecture for streaming sources, covering the value of streaming with historical data, what Zoomdata didn’t think about the first time around, and the new architecture for streaming analytics, from the user experience to the details of the implementation. This modern streaming architecture builds a table on the fly based on the shape of the data, writes as a stream to that table, then pushes the new data to visualizations without complex reaggregations, enabling users to view real-time metrics side by side with their historical data quickly and efficiently.

Photo of Erin Recachinas

Erin Recachinas

Zoomdata

Erin Recachinas is a software engineer and Scrum Master for the infrastructure and performance team at Zoomdata, where she and her team recently rearchitected Zoomdata’s streaming capabilities. Previously, Erin was a full stack engineer at Appian. She studied computer science and mathematics at the University of Virginia.