About Space Invaders and automated scaling





Who is this presentation for?
- Software engineers who want to understand the possibilities and power of reinforcement learning.
Level
BeginnerDescription
Adobe Cloud Platform is the core component of Adobe’s cloud offerings, serving billions of requests per day. Fast reaction time to load spikes and cost-efficient scaling are essential. Therefore, Adobe is constantly evaluating autoscaling algorithms to incrementally improve efficiency.
Deep reinforcement learning (RL) success stories raise hopes for building a scaling algorithm that can learn from historical load data while still being able to react to unusual load spikes in an appropriate way.
Michael Friedrich and Stefanie Grunwald took an algorithm that successfully learns Space Invaders and other Atari Games and applied it to service autoscaling. If a RL algorithm can learn how to play Space Invaders, why shouldn’t it learn how to automatically scale IT services?
Join in to learn how they built an OpenAI Gym using TensorFlow to train a neural network based on historical load data. You’ll deep dive into the technical details of the changes required to adapt the algorithm to the new domain of scaling and see how the next generation of flight controllers for racing drones helped them to improve their results—and why flight controllers solve a similar problem as autoscaling mechanisms.
Prerequisite knowledge
- A basic understanding of machine learning and AI concepts
What you'll learn
- Learn autoscaling techniques for cloud services and their pros and cons
- Understand how to train an AI with reinforcement learning and OpenAI Gym and how to adapt existing RL algorithms to any new domain

Michael Friedrich
Adobe
Michael Friedrich is a Senior Computer Scientist for the Adobe Cloud Platform at Adobe. His team brings cloud operations to a new level, using machine learning to automate complex development and delivery processes, including by implementing automated canary analysis for deployments or researching new automated scaling solutions.
Prior to Adobe he was the chief software engineer for Hamburg Süd (now part of Maersk) and was working on their container routing software.

Stefanie Grunwald
Adobe
Stefanie Grunwald is a senior data and platform engineer with Adobe Experience Cloud. As a trained software architect, she’s been working in the field of data science and data engineering since 2011, applying the best practices from software engineering to building intelligent data platforms. With her heart devoted to DataOps and the OSS community, she and her team help others at Adobe become data-driven through automation and real-time insights.
Presented by
Elite Sponsors
Strategic Sponsor
Exabyte Sponsor
Impact Sponsor
Contact us
confreg@oreilly.com
For conference registration information and customer service
partners@oreilly.com
For more information on community discounts and trade opportunities with O’Reilly conferences
aisponsorships@oreilly.com
For information on exhibiting or sponsoring a conference
pr@oreilly.com
For media/analyst press inquires