Computational devices and systems are constantly telling their stories in the form of logs: logins and logouts, network connections and disconnections, sensor readings, customer interactions, database activities, configuration changes, and application errors. These logs can often tell us important information about the operational health, security, and usage patterns of our systems.
However, the scale and variety of raw log data can pose a formidable challenge. Logs come in many different formats, and these formats themselves evolve over time. For large log volumes, interactive single-machine processing is not an option. Random fluctuations and transient disruptions can make static rules and thresholds prone to “false positive” alerts.
In this talk we will discuss a few ways in which machine learning techniques can be combined with human guidance in order to understand what the logs are telling us:
We’ll also give examples of interesting findings uncovered by these applying these approaches to large volumes of real data in a production log-management service.
David Andrzejewski is Lead Data Sciences Engineer at Sumo Logic, which he joined in 2011 after a postdoctoral research position working on knowledge discovery at Lawrence Livermore National Laboratory (LLNL). He is interested in developing tools that combine the power of machine learning with human insights, and has published work applying these ideas to problems in biomedical text mining, information retrieval and software behavior. David completed his PhD in Computer Sciences at the University of Wisconsin-Madison in 2010, where he had also previously received an M.S. in Computer Sciences and a B.S. in Computer Engineering, Mathematics and Computer Sciences.
Comments on this page are now closed.
For exhibition and sponsorship opportunities, contact Susan Stewart at email@example.com
For information on trade opportunities with O'Reilly conferences, email firstname.lastname@example.org
For media-related inquiries, contact Maureen Jennings at email@example.com
View a complete list of Strata contacts