Deep learning meets Kubernetes: Running massively parallel inference pipelines efficiently
Who is this presentation for?Data engineers, data architects, developers
Nearmap captures terabytes of aerial imagery daily. With the introduction of AI capabilities, Nearmap has leveraged Kubernetes to generate AI content based on tens of petabytes of images effectively and efficiently. Suneeta Mall walks you through the joys of building and running this system at scale, challenges encountered, the company’s resolution, and future work. Some of the challenges encountered, for instance, have been around provisioning a conditionally stateful fault-tolerant directed acyclic graph (DAG), autoscaling, etcd, networking, GPU orchestration and sharing, and hybrid cluster setup.
Exploiting Kubernetes’s resilient, highly available, extensible, declarative state management capabilities, the company built a fully automated solution that at the core is a DAG—with each node fanning out to achieve said task in shortest time possible. The company uses this architecture for a deep learning inference of billions of high-resolution aerial images on a mix of GPU and CPU compute, primarily driven off the spot pricing scheme. You’ll discover the architecture of this autoscalable solution that exhausted all K80 spot GPUs across all US data centers of AWS for weeks.
Built purely using open source software, this solution is so resilient and elastic that the company has scaled on demand from zero to thousands of compute nodes, crunching through petabytes of images to generate semantic segmentation results with effectively no manual intervention. This system has already produced semantic content on over a million-kilometers-squared area at resolution as high as 5 cm per pixel in just two weeks.
- A basic understanding of Kubernetes and machine learning
What you'll learn
- Learn how to use Kubernetes core constructs to build massively parallel deep learning inference pipelines
- Discover how Kubernetes can be used to run machine learning at scale, including how to exploit features such as autoscaling and run workloads purely on spot and preemptible virtual machines
- Hear new ideas about how to use thousands of GPUs simultaneously and obtain statistics around GPU use
- Understand the challenges encountered in running Kubernetes at scale and their resolutions
Suneeta Mall is a senior data scientist at Nearmap, where she leads the engineering efforts of the Artificial Intelligence Division. She led the efforts of migrating Nearmap’s backend services to Kubernetes. In her 12 years of software industry experience, she’s worked on a solving variety of challenging technical and business problems in the field of big data, machine learning, GIS, travel, DevOps, and telecommunication. She earned her PhD from University of Sydney and bachelor’s of computer science and engineering.
Leave a Comment or Question
Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?
Join the conversation here (requires login)
Premier Diamond Sponsors
Premier Exhibitor Plus
For conference registration information and customer service
For more information on community discounts and trade opportunities with O’Reilly conferences
For information on exhibiting or sponsoring a conference
For media/analyst press inquires