Build Systems that Drive Business
Sep 30–Oct 1, 2018: Training
Oct 1–3, 2018: Tutorials & Conference
New York, NY

Bulk image processing using Kubernetes

Mike Newswanger (Elastic)
3:50pm–4:30pm Tuesday, October 2, 2018
Distributed Systems, How We Built It
Location: Gramercy Level: Intermediate
Secondary topics:  Systems Architecture & Infrastructure

Prerequisite knowledge

  • Familiarity with Kubernetes (useful but not required)

What you'll learn

  • Learn how Kubernetes and Google Cloud were used to burst and extend the capacity of a physical infrastructure for optimizing almost 10 million images in less than two weeks

Description

Mike Newswanger explains how he used Kubernetes and Google Cloud to burst and extend the capacity of a physical infrastructure for optimizing almost 10 million images in less than two weeks. The process was running 3,600 parallel streams with near 100% compute utilization. Mike offers an overview of the infrastructure design, implementation process, performance analysis, and problems encountered along the way. He also shares real-world examples of how cloud compute resources and containers can boost the throughput of a traditional organization.

Photo of Mike Newswanger

Mike Newswanger

Elastic

Mike Newswanger is a cloud engineer at Elastic. Previously, he was a site reliability engineer at Stack Overflow. Mike enjoys solving problems, particularly those related to scale, and tackling the challenges of creating secure, performant, resilient, and maintainable code and infrastructure. Over his career, he’s worked in all parts of the application development lifecycle, from inception to deployment.