Mar 15–18, 2020

Deep learning meets Kubernetes: Running massively parallel inference pipelines efficiently

Suneeta Mall (Nearmap)
1:45pm2:25pm Tuesday, March 17, 2020
Location: LL20D

Who is this presentation for?

Data engineers, data architects, developers

Level

Intermediate

Description

Nearmap captures terabytes of aerial imagery daily. With the introduction of AI capabilities, Nearmap has leveraged Kubernetes to generate AI content based on tens of petabytes of images effectively and efficiently. Suneeta Mall walks you through the joys of building and running this system at scale, challenges encountered, the company’s resolution, and future work. Some of the challenges encountered, for instance, have been around provisioning a conditionally stateful fault-tolerant directed acyclic graph (DAG), autoscaling, etcd, networking, GPU orchestration and sharing, and hybrid cluster setup.

Exploiting Kubernetes’s resilient, highly available, extensible, declarative state management capabilities, the company built a fully automated solution that at the core is a DAG—with each node fanning out to achieve said task in shortest time possible. The company uses this architecture for a deep learning inference of billions of high-resolution aerial images on a mix of GPU and CPU compute, primarily driven off the spot pricing scheme. You’ll discover the architecture of this autoscalable solution that exhausted all K80 spot GPUs across all US data centers of AWS for weeks.

Built purely using open source software, this solution is so resilient and elastic that the company has scaled on demand from zero to thousands of compute nodes, crunching through petabytes of images to generate semantic segmentation results with effectively no manual intervention. This system has already produced semantic content on over a million-kilometers-squared area at resolution as high as 5 cm per pixel in just two weeks.

Prerequisite knowledge

  • A basic understanding of Kubernetes and machine learning

What you'll learn

  • Learn how to use Kubernetes core constructs to build massively parallel deep learning inference pipelines
  • Discover how Kubernetes can be used to run machine learning at scale, including how to exploit features such as autoscaling and run workloads purely on spot and preemptible virtual machines
  • Hear new ideas about how to use thousands of GPUs simultaneously and obtain statistics around GPU use
  • Understand the challenges encountered in running Kubernetes at scale and their resolutions
Photo of Suneeta Mall

Suneeta Mall

Nearmap

Suneeta Mall is a senior data scientist at Nearmap, where she leads the engineering efforts of the Artificial Intelligence Division. She led the efforts of migrating Nearmap’s backend services to Kubernetes. In her 12 years of software industry experience, she’s worked on a solving variety of challenging technical and business problems in the field of big data, machine learning, GIS, travel, DevOps, and telecommunication. She earned her PhD from University of Sydney and bachelor’s of computer science and engineering.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)

Contact us

confreg@oreilly.com

For conference registration information and customer service

partners@oreilly.com

For more information on community discounts and trade opportunities with O’Reilly conferences

Become a sponsor

For information on exhibiting or sponsoring a conference

pr@oreilly.com

For media/analyst press inquires