The ability to make quick decisions based on real-time data is important, but most business decisions aren’t made solely on a “window” of the most recent information. Analysts, executives, and decision makers everywhere want historical trends so they can look at what’s happening now in the context of historical data.
Big data application Zoomdata brought streaming and historical data together, addressing the pie chart problem. However, streaming in the age of the IoT requires handling of data in volumes not seen in traditional feeds, and Zoomdata’s legacy streaming architecture wasn’t extensible enough to meet the emerging and future needs of this evolving space.
Erin Recachinas explains how Zoomdata moved to a scalable microservice architecture for streaming sources, covering the value of streaming with historical data, what Zoomdata didn’t think about the first time around, and the new architecture for streaming analytics, from the user experience to the details of the implementation. This modern streaming architecture builds a table on the fly based on the shape of the data, writes as a stream to that table, then pushes the new data to visualizations without complex reaggregations, enabling users to view real-time metrics side by side with their historical data quickly and efficiently.
Erin Recachinas is a software engineer and Scrum Master for the infrastructure and performance team at Zoomdata, where she and her team recently rearchitected Zoomdata’s streaming capabilities. Previously, Erin was a full stack engineer at Appian. She studied computer science and mathematics at the University of Virginia.
©2018, O’Reilly UK Ltd • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • firstname.lastname@example.org