Sep 9–12, 2019

Live coding a self-driving car (without a car)

Paris Buttfield-Addison (Secret Lab), Mars Geldard (University of Tasmania), Tim Nugent (
4:50pm5:30pm Thursday, September 12, 2019
Location: LL21 C/D

Who is this presentation for?

Programmers, AI product managers, anyone really!




Are you a scientist who wants to test a research problem without building costly and complicated real-world rigs? A self-driving car engineer who wants to test their AI logic in a constrained virtual world? A data scientist who needs to solve a thorny real-world problem without touching a production environment? Have you considered AI problem solving using game engines? No? We’ll teach you how. Live on stage. No slides.

This session explore ML and AI problem solving with game engines. Learn how you could use a game engine to train, explore, and manipulate intelligence agents that learn.

Game engines are a great place to explore ML and AI. They’re wonderful constrained problem spaces, tiny little ecosystems for you to explore a problem in. Learn how to use them even though you’re not a game developer.

In this session, we’ll look at, via Live Coding (!):

  • how video game engines are a perfect environment to constrain a problem and train an agent
  • how easy it is to get started, using the Unity engine and Google’s TensorFlow for Python
  • how to build up a model, and use it in the engine, to explore a particular idea or problem
  • PPO (proximal policy optimisation) for generic but useful machine learning
  • how deep reinforcement learning using virtual environments is the future of ML

This session is for non-game developers to learn how they can use game technologies to further their understanding of machine learning fundamentals, and solve problems using a combination of open source tools, such as TensorFlow and PyTorch, and game engines, such as Unity.

See how a quadruped can be trained to walk, or explore, fetch, and manipulate a virtual world! Train a car to drive! Teach a disembodied hand to play tennis! We’ll do it all, live on stage.

It’s a bit technical, a bit creative, and it’s all on the cutting edge.

Prerequisite knowledge

Nothing special. Basic AI principles.

What you'll learn

The takeaway is that game engines, such as Unity, are a great 'biosphere' for testing complex AI problems in a cost effective, engaging, possibly visual way. This session will show how to get started.
Photo of Paris Buttfield-Addison

Paris Buttfield-Addison

Secret Lab

Paris Buttfield-Addison is cofounder of Secret Lab, a game development studio based in beautiful Hobart, Australia. Secret Lab builds games and game development tools, including the multi-award-winning ABC Play School iPad games, BAFTA-winning Night in the Woods, the Qantas airlines Joey Playbox games, and the Yarn Spinner narrative game framework. Previously, Paris was mobile product manager for Meebo (acquired by Google). Paris particularly enjoys game design, statistics, the blockchain, machine learning, and human-centered technology research and writes technical books on mobile and game development (more than 20 so far) for O’Reilly Media (currently working on ‘Head First Swift’ and ‘Practical AI with Swift’). He holds a degree in medieval history and a PhD in computing.

Photo of  Mars Geldard

Mars Geldard

University of Tasmania

Marina Rose Geldard, more commonly known as Mars, is a researcher from Down Under in Tasmania. Entering the world of technology relatively late as a mature-age student, she has found her place in the world: an industry where she can apply her lifelong love of mathematics and optimization. When she is not busy being the most annoyingly eager researcher ever, she compulsively volunteers at industry events, dabbles in research, and serves on the executive committee for her state’s branch of the Australian Computer Society (ACS). She’s currently writing “Practical AI with Swift” for O’Reilly Media.

Photo of Tim Nugent

Tim Nugent

Tim Nugent pretends to be a mobile app developer, game designer, tools builder, researcher, and tech author. When he isn’t busy avoiding being found out as a fraud, Tim spends most of his time designing and creating little apps and games he won’t let anyone see. He also spent a disproportionately long time writing his tiny little bio, most of which was taken up trying to stick a witty sci-fi reference in…before he simply gave up.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)

Contact us

For conference registration information and customer service

For more information on community discounts and trade opportunities with O’Reilly conferences

Become a sponsor

For information on exhibiting or sponsoring a conference

Contact list

View a complete list of O'Reilly AI contacts