14–17 Oct 2019

About Space Invaders and automated scaling

13:4514:25 Wednesday, 16 October 2019
Location: Blenheim Room - Palace Suite
Average rating: ****.
(4.67, 3 ratings)

Who is this presentation for?

  • Software engineers who want to understand the possibilities and power of reinforcement learning.




Adobe Cloud Platform is the core component of Adobe’s cloud offerings, serving billions of requests per day. Fast reaction time to load spikes and cost-efficient scaling are essential. Therefore, Adobe is constantly evaluating autoscaling algorithms to incrementally improve efficiency.

Deep reinforcement learning (RL) success stories raise hopes for building a scaling algorithm that can learn from historical load data while still being able to react to unusual load spikes in an appropriate way.

Michael Friedrich and Stefanie Grunwald took an algorithm that successfully learns Space Invaders and other Atari Games and applied it to service autoscaling. If a RL algorithm can learn how to play Space Invaders, why shouldn’t it learn how to automatically scale IT services?

Join in to learn how they built an OpenAI Gym using TensorFlow to train a neural network based on historical load data. You’ll deep dive into the technical details of the changes required to adapt the algorithm to the new domain of scaling and see how the next generation of flight controllers for racing drones helped them to improve their results—and why flight controllers solve a similar problem as autoscaling mechanisms.

Prerequisite knowledge

  • A basic understanding of machine learning and AI concepts

What you'll learn

  • Learn autoscaling techniques for cloud services and their pros and cons
  • Understand how to train an AI with reinforcement learning and OpenAI Gym and how to adapt existing RL algorithms to any new domain
Photo of Michael Friedrich

Michael Friedrich


Michael Friedrich is a Senior Computer Scientist for the Adobe Cloud Platform at Adobe. His team brings cloud operations to a new level, using machine learning to automate complex development and delivery processes, including by implementing automated canary analysis for deployments or researching new automated scaling solutions.

Prior to Adobe he was the chief software engineer for Hamburg Süd (now part of Maersk) and was working on their container routing software.

Photo of Stefanie Grunwald

Stefanie Grunwald


Stefanie Grunwald is a senior data and platform engineer with Adobe Experience Cloud. As a trained software architect, she’s been working in the field of data science and data engineering since 2011, applying the best practices from software engineering to building intelligent data platforms. With her heart devoted to DataOps and the OSS community, she and her team help others at Adobe become data-driven through automation and real-time insights.

  • Intel AI
  • O'Reilly
  • Amazon Web Services
  • IBM Watson
  • Dell Technologies
  • Hewlett Packard Enterprise
  • AXA

Contact us


For conference registration information and customer service


For more information on community discounts and trade opportunities with O’Reilly conferences


For information on exhibiting or sponsoring a conference


For media/analyst press inquires