Integrating with heterogeneous storage in a cloud native environment has always been a challenge, and detecting problems and fixing them in a timely fashion is important for mission-critical workloads.
Quinton Hoole examines a common volume metrics model designed to retrieve data from heterogeneous storage in a cloud native environment. Quinton also details an ML module that analyzes the data to detect anomalies and explains how it helps identify problems early to keep the storage systems healthy. Volume metrics such as IOPS, bandwidth, latency, and capacity are generated from storage backends serving workloads running on Kubernetes and collected by the Prometheus server. Data is piped through Kafka and then parsed and saved in MongoDB. The ML module retrieves data to train the models and chooses the best model to detect anomalous data points.
This session is sponsored by Futurewei.
Quinton Hoole is the technical vice president of R&D at Futurewei. An engineering leader, system architect, designer, and software developer with 20+ years of experience across a variety of industries and technology bases, including cloud computing, machine learning, online advertising, telecoms, financial services, and ecommerce, his specialties are cloud computing, large system architecture, and open source solutions. Previously, he was a founding engineer at Amazon EC2, tech lead at Google/CNCF Kubernetes, and tech lead and manager at Google Ads serving SRE.
©2019, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • email@example.com