Sep 9–12, 2019

Build a self-driving car without a car! ML problem-solving with a game engine

Paris Buttfield-Addison (Secret Lab), Tim Nugent (, Mars Geldard (University of Tasmania)
9:00am12:30pm Tuesday, September 10, 2019
Location: 231

Who is this presentation for?

Programmers, team leaders, and those wanting to do AI projects that can't afford to (or don't want to commit to) building a whole car (or similar!)




Are you a scientist who wants to test a research problem without building costly and complicated real-world rigs? A self-driving car engineer who wants to test their AI logic in a constrained virtual world? A data scientist who needs to solve a thorny real-world problem without touching a production environment? Have you considered simulation-driven ML problem solving with a game engine?

Imagine you’re building a high-volume, expensive, robot-driven warehouse. Your pick, place, and packing robots need to get to the right place quickly, find the right item and sort it to the right place without colliding with each other, the shelves, or people. But you don’t have any robots yet, and you need to start writing, or training, the behavior.

A game engine is a controlled, self-contained spatial, physical environment that (can) closely replicate (enough of) the real world (to be useful).

This tutorial will explore how you can use game engines to explore and solve problems——such as the aforementioned robot-driven warehouse——in a simulated environment, without building costly and complicated real-world rigs.

We will teach you how to solve AI and ML problems using the Unity game engine, and TensorFlow for Python. Learn how you could use a game engine to train, explore, and manipulate intelligent agents that learn, explore, and help you understand a problem space.

Game engines are a great place to explore ML and AI. They’re wonderful constrained problem spaces, tiny little ecosystems for you to explore a problem in. Here you can learn how to use them even though you’re not a game developer.

In this tutorial, we’ll look at:

  • how video game engines are a perfect environment to constrain a problem and train an agent
  • how easy it is to get started, using the Unity engine and Google’s TensorFlow for Python
  • how to build up a model, and use it in the engine, to explore a particular idea or problem
  • PPO (proximal policy optimisation) for generic but useful machine learning
    deep reinforcement learning, and how it lets you explore and study complex behaviours

Specifically, we will:

  • teach the very basics of the Unity game engine
  • explore how to design and setup a scene in Unity for both training and use of a ML model
  • show how to train a model, using TensorFlow, using the Unity scene
  • explore how you pick the right actions, observations, and rewards for training
  • show the difference between imitation learning, reinforcement learning, and other approaches
  • discuss the use of the trained model, and potential applications
  • show you how to train AI agents in complicated scenarios and make the real world better by leveraging the virtual

We’ll explore fun, engaging scenarios, including virtual self-driving cars, bipedal human-like walking robots, disembodied hands that can play tennis, and packing a warehouse efficiently using robots.

This tutorial is for non-game developers to learn how they can use game technologies to further their understanding of machine learning fundamentals, and solve problems using a combination of open source ML tools and game engines. Deep reinforcement learning using virtual environments is the beginning of an exciting new wave of AI, and game engines are on the leading edge! You’ll learn how to approach your ML work in a totally new way.

We’ll be using Unity, TensorFlow, and related tools, but all the principles and skills are general and transferrable to other game engines or simulation platforms (e.g. Open AI, and the like).

It’s a bit technical, a bit creative, and you’ll leave with brand-new skills and an enthusiasm for using them. Participants will leave with their own, trained, virtual robot (or car, or animal, or… anything they can imagine!)

Prerequisite knowledge

Nothing special. A basic, basic understanding of ML-concepts would be useful, but nowhere near essential! Basic Python would probably ease progress in the tutorial, too, but is _not_ essential at all. Unity knowledge would make it all easier, but isn't needed at all.

Materials or downloads needed in advance

We'll supply a download in advance, and simple instructions. Unity is a free download, and TensorFlow is available from common package managers. All the requirements run on Windows, macOS, and Linux.

What you'll learn

Basic Unity. Basic TensorFlow. Advanced Unity ML. Combining the Unity and TensorFlow for amazing ML- and AI-powered explorations, training, and testing. How to use a game engine (tutorial will explore Unity + Python TensorFlow, but principles are generally applicable) to test out and explore ML- and AI-problems that have a visual or physical component (e.g. self-driving cars, real-world interactions, etc.)
Photo of Paris Buttfield-Addison

Paris Buttfield-Addison

Secret Lab

Paris Buttfield-Addison is cofounder of Secret Lab, a game development studio based in beautiful Hobart, Australia. Secret Lab builds games and game development tools, including the multi-award-winning ABC Play School iPad games, the BAFTA- and IGF-winning Night in the Woods, the Qantas airlines Joey Playbox games, and the Yarn Spinner narrative game framework. Previously, Paris was mobile product manager for Meebo (acquired by Google). Paris particularly enjoys game design, statistics, the blockchain, machine learning, and human-centered technology research and writes technical books on mobile and game development (more than 20 so far) for O’Reilly. He holds a degree in medieval history and a PhD in computing.

Photo of Tim Nugent

Tim Nugent

Tim Nugent pretends to be a mobile app developer, game designer, tools builder, researcher, and tech author. When he isn’t busy avoiding being found out as a fraud, Tim spends most of his time designing and creating little apps and games he won’t let anyone see. He also spent a disproportionately long time writing his tiny little bio, most of which was taken up trying to stick a witty sci-fi reference in…before he simply gave up.

Photo of  Mars Geldard

Mars Geldard

University of Tasmania

Marina Rose Geldard, more commonly known as Mars, is a researcher from Down Under in Tasmania. Entering the world of technology relatively late as a mature-age student, she has found her place in the world: an industry where she can apply her lifelong love of mathematics and optimization. When she is not busy being the most annoyingly eager researcher ever, she compulsively volunteers at industry events, dabbles in research, and serves on the executive committee for her state’s branch of the Australian Computer Society (ACS). She’s currently writing “Practical AI with Swift” for O’Reilly Media.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)

Contact us

For conference registration information and customer service

For more information on community discounts and trade opportunities with O’Reilly conferences

Become a sponsor

For information on exhibiting or sponsoring a conference

Contact list

View a complete list of O'Reilly AI contacts