Streaming data generates data continuously by thousands of data sources, typically sending in the data records simultaneously and in small sizes (on the order of bytes or kilobytes). Streaming data includes a wide variety of data such as sensors connected to the internet of things (IoT), log files generated by customers using mobile or web applications, ecommerce purchases, in-game player activity, information from social networks, financial trading floors, or geospatial services, and more.
Ingesting and collecting, storing, and processing billions of events per second in real time is not a simple process. The challenges are in both transforming the raw data to metrics and automatically analyzing the metrics in real time to produce the insights. Roy Ben-Alta and Ira Cohen detail various design patterns and share a solution implemented using Amazon Kinesis as a real-time event data processing pipeline that feeds Anodot’s AI-based analytics service, discovering and alerting on the anomalies in the data in real time and helping you avoid costly business incidents.
Roy Ben-Alta is a solution architect and principal business development manager at Amazon Web Services, where he focuses on AI and real-time streaming technologies and working with AWS customers to build data-driven products (whether batch or real time) and create solutions powered by ML in the cloud. Roy has worked in the data and analytics industry for over a decade and has helped hundreds of customers bring compelling data-driven products to the market. He serves on the advisory board of Applied Mathematics and Data Science at Post University in Connecticut. Roy holds a BSc in information systems and an MBA from the University of Georgia.
Ira Cohen is a cofounder and chief data scientist at Anodot, where he’s responsible for developing and inventing the company’s real-time multivariate anomaly detection algorithms that work with millions of time series signals. He holds a PhD in machine learning from the University of Illinois at Urbana-Champaign and has over 12 years of industry experience.
©2018, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • firstname.lastname@example.org