Fueling innovative software
July 15-18, 2019
Portland, OR

Building telemetry and anomaly-detection models for cloud native storage (sponsored by Futurewei)

Quinton Hoole (Futurewei)
1:45pm2:25pm Thursday, July 18, 2019
Sponsored
Location: F150

Level

Intermediate

Integrating with heterogeneous storage in a cloud native environment has always been a challenge, and detecting problems and fixing them in a timely fashion is important for mission-critical workloads.

Quinton Hoole examines a common volume metrics model designed to retrieve data from heterogeneous storage in a cloud native environment. Quinton also details an ML module that analyzes the data to detect anomalies and explains how it helps identify problems early to keep the storage systems healthy. Volume metrics such as IOPS, bandwidth, latency, and capacity are generated from storage backends serving workloads running on Kubernetes and collected by the Prometheus server. Data is piped through Kafka and then parsed and saved in MongoDB. The ML module retrieves data to train the models and chooses the best model to detect anomalous data points.

This session is sponsored by Futurewei.

Prerequisite knowledge

  • A basic understanding of cloud native, Kubernetes, and databases

What you'll learn

  • Learn about a common metrics model designed to retrieve data from heterogenous storage in a cloud native environment
Photo of Quinton Hoole

Quinton Hoole

Futurewei

Quinton Hoole is the technical vice president of R&D at Futurewei. An engineering leader, system architect, designer, and software developer with 20+ years of experience across a variety of industries and technology bases, including cloud computing, machine learning, online advertising, telecoms, financial services, and ecommerce, his specialties are cloud computing, large system architecture, and open source solutions. Previously, he was a founding engineer at Amazon EC2, tech lead at Google/CNCF Kubernetes, and tech lead and manager at Google Ads serving SRE.