Presented By O’Reilly and Intel Nervana
Put AI to work
September 17-18, 2017: Training
September 18-20, 2017: Tutorials & Conference
San Francisco, CA

End-to-end deep learning at the edge

1:45pm–2:25pm Wednesday, September 20, 2017
Implementing AI
Location: Imperial A Level: Beginner
Secondary topics:  Data science and AI, Deep learning, IoT (including smart cities, manufacturing, smart homes/buildings), Transportation and autonomous vehicles
Average rating: ****.
(4.00, 1 rating)

Prerequisite Knowledge

  • Familiarity with deep learning and convolution networks
  • A basic understanding of domain adaptation, transfer learning, cross-modality training, and long short-term memory models (useful but not required)

What you'll learn

  • Understand a method for building an end-to-end driving policy to leverage the 10 trillion miles driven every year, using a network of connected devices


The robustness of end-to-end driving policy models depends on having access to the largest possible training dataset, exposing the true diversity of the 10 trillion miles that humans drive every year in the real world. However, current approaches are limited to models trained using homogenous data from a small number of vehicles running in controlled environments or in simulation, which fail to perform adequately in real-world dangerous corner cases. Safe driving requires continuously resolving a long tail of those corner cases. The only possible way to train a robust driving policy model is therefore to continuously capture as many of these cases as possible. The capture of driving data is unfortunately constrained by the reduced compute capabilities of the devices running at the edge and the limited network connectivity to the cloud, making the task of building robust end-to-end driving policies very complex.

Bruno Fernandez-Ruiz offers an overview of a network of connected devices deployed at the edge running deep learning models that continuously capture, select, and transfer to the cloud “interesting” monocular camera observations, vehicle motion, and driver actions. The collected data is used to train an end-to-end vehicle driving policy, which also guarantees that the information gain of the learned model is monotonically increasing, effectively becoming progressively more selective of the data captured by the edge devices as it walks down the tail of corner cases.

Photo of Bruno Fernandez-Ruiz

Bruno Fernandez-Ruiz


Bruno Fernandez-Ruiz is cofounder and CTO at Nexar, where he and his team are using large-scale machine learning and machine vision to capture and analyze millions of sensor and camera readings in order to make our roads safer. Previously, Bruno was a senior fellow at Yahoo, where he oversaw the development and delivery of Yahoo’s personalization, ad targeting, and native advertising teams; his prior roles at Yahoo included chief architect for Yahoo’s cloud and platform and chief architect for international. Prior to joining Yahoo, Bruno founded OneSoup (acquired by Synchronica and now part of the Myriad Group) and YamiGo; was an enterprise architect for Fidelity Investments; served as manager in Accenture’s Center for Strategic Research Group, where he cofounded Meridea Financial Services and Accenture’s claim software solutions group. Bruno holds an MSc in operations research and transportation science from MIT, with a focus on intelligent transportation systems.