Tesla calculated that its fleet has to drive 10 billion kilometers to train and calibrate its machine-learning algorithms to drive autonomously. The challenge for autonomous driving, however, is that the algorithms can hardly be initially trained with all possible driving conditions—due to the highly dynamic runtime environment for the sensors—and dangerous situations only occur at a very low frequency. This problem cannot be solved by increasing the amount of training data, since there will be driving situations where no data will be available due to disturbed or noisy sensor output. It requires a monitoring system able to detect regions where the probability is high that sensors will fail.
To overcome this problem, a system has to be created where cars, regardless of their brand, sample their environment during operation and dangerous situations are detected by a central server and propagated to other cars in real time, a process much like website monitoring. Michael Nolting explores how such a system might be realized (with a real-time architecture based on Kafka, Flink, Elasticsearch, and Kibana) and how its requirements can be fulfilled. Michael shares a generalized method for detecting anomalies (dangerous situations) based on bootstrapping, which is able to operate on non-Gaussian distributions. Bootstrapping allows assigning measures of accuracy (defined in terms of bias, variance, confidence intervals, prediction error, or some other such measure) to sample estimates and enables estimation of the sampling distribution of almost any statistic using random sampling methods
Michael Nolting is a data scientist for Volkswagen commercial vehicles. Michael has worked in a variety of research fields at Volkswagen AG, including adapting big data technologies and machine learning algorithms to the automotive context. Previously, he was head of a big data analytics team at Sevenval Technologies. Michael holds a Dipl.-Ing. degree in electrical engineering and an MSc degree in computer science, both from the Technical University of Brunswick in Germany, and a PhD in computer science.
©2017, O’Reilly UK Ltd • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • firstname.lastname@example.org