Presented By
O’Reilly + Intel AI
Put AI to Work
April 15-18, 2019
New York, NY

Using topological data analysis to understand, build, and improve neural networks

Gunnar Carlsson (Ayasdi)
9:00am12:30pm Tuesday, April 16, 2019
Models and Methods
Location: Beekman
Secondary topics:  Deep Learning and Machine Learning tools, Models and Methods

Who is this presentation for?

CDO, Data Scientist, Data Engineering

Level

Advanced

Prerequisite knowledge

A solid understanding of neural networks will be required. Familiarity with MNIST and SVHN will facilitate understanding.

Materials or downloads needed in advance

N/A

What you'll learn

1. Methods for determining how and what neural nets learn 2. Methods for dramatically improving the performance and accuracy of neural nets 3. A framework for generalizing and adapting efficient architectures for neural nets based on the underlying data

Description

Neural networks have demonstrated a great deal of success in the study of various kinds of data, including images, text, time series, and many others. One issue that restricts their applicability, however, is the fact that it is not understood in any kind of detail how they work. A related problem is that there is often a certain kind of overfitting to particular data sets, which results in the possibility of adversarial behavior. For these reasons, it is very desirable to develop methods for developing some understanding of the internal states of the neural networks. Because of the very large number of nodes (or neurons) in the networks, this becomes a problem in data analysis, specifically for unsupervised data analysis.

This talk will discuss how topological data analysis can be used to obtain insight into the working of neural networks. Examples are drawn from networks trained on image data sets, but topological modeling can just as easily explain the workings of many other neural networks.

For example, specific classes of architectures have been developed to address specific data types, in particular we have the convolutional architectures. These constructions appear to be one-offs for particular classes of problems.

The talk will address how to take much more general data types and adapt the architecture to them. Specific examples will demonstrate cases where one can construct an architecture for a single data set – a custom neural network. This discussion will be preceded by a discussion of how generalized architectures.

The talk will conclude with a discussion of how TDA can be used to persist derived or learned features. Where applied, this approach delivers a speedup on the MNIST dataset of a little under 2X. For the more complicated SVHN dataset, the performance is even more substantial, roughly 3.5X until hitting the .8 threshold, and lowering to 2.5-3X thereafter. An examination of the graphs of validation accuracy vs. number of batch iterations suggests, plotted to 30,000 iterations, that at the higher accuracy ranges, the standard method may never attain the results of the TDA boosted method.

The implications of this work are broad and considerable as it illuminates the black box of neural nets while simultaneously improving the performance and, finally, creating a roadmap for even more performance enhancements. Just the first finding (how and what NNs learn) will have broad operational implications for data.

Photo of Gunnar Carlsson

Gunnar Carlsson

Ayasdi

Gunnar Carlsson is a professor of mathematics (emeritus) at Stanford University and is cofounder and president at Ayasdi, which is commercializing products based on machine intelligence and topological data analysis. Gunnar has spent his career devoted to the study of topology, the mathematical study of shape. Originally, his work focused on the pure aspects of the field, but in 2000 he began work on the applications of topology to the analysis of large and complex datasets, which led to a number of projects, notably a multi-university initiative funded by the Defense Advanced Research Projects Agency. He has taught at the University of Chicago, the University of California, San Diego, Princeton University, and, since 1991, Stanford University, where he has served as the chair of the Mathematics Department. He is also a founder of the ATMCS series of conferences focusing on the applications of topology, and is a founding editor of the Journal for Applied and Computational Topology. Gunnar is the author of over 100 academic papers and has given numerous addresses to scholarly meetings. He holds a BA in mathematics from Harvard and a PhD in mathematics from Stanford. He is married with three grown children.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)