The robustness of end-to-end driving policy models depends on having access to the largest possible training dataset, exposing the true diversity of the 10 trillion miles that humans drive every year in the real world. However, current approaches are limited to models trained using homogenous data from a small number of vehicles running in controlled environments or in simulation, which fail to perform adequately in real-world dangerous corner cases. Safe driving requires continuously resolving a long tail of those corner cases. The only possible way to train a robust driving policy model is therefore to continuously capture as many of these cases as possible. The capture of driving data is unfortunately constrained by the reduced compute capabilities of the devices running at the edge and the limited network connectivity to the cloud, making the task of building robust end-to-end driving policies very complex.
Bruno Fernandez-Ruiz offers an overview of a network of connected devices deployed at the edge running deep learning models that continuously capture, select, and transfer to the cloud “interesting” monocular camera observations, vehicle motion, and driver actions. The collected data is used to train an end-to-end vehicle driving policy, which also guarantees that the information gain of the learned model is monotonically increasing, effectively becoming progressively more selective of the data captured by the edge devices as it walks down the tail of corner cases.
Bruno Fernandez-Ruiz is cofounder and CTO at Nexar, where he and his team are using large-scale machine learning and machine vision to capture and analyze millions of sensor and camera readings in order to make our roads safer. Previously, Bruno was a senior fellow at Yahoo, where he oversaw the development and delivery of Yahoo’s personalization, ad targeting, and native advertising teams; his prior roles at Yahoo included chief architect for Yahoo’s cloud and platform and chief architect for international. Prior to joining Yahoo, Bruno founded OneSoup (acquired by Synchronica and now part of the Myriad Group) and YamiGo; was an enterprise architect for Fidelity Investments; served as manager in Accenture’s Center for Strategic Research Group, where he cofounded Meridea Financial Services and Accenture’s claim software solutions group. Bruno holds an MSc in operations research and transportation science from MIT, with a focus on intelligent transportation systems.
©2017, O'Reilly Media, Inc. • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • firstname.lastname@example.org