Mar 15–18, 2020

Log anomaly detector with NLP and unsupervised machine learning

Zak Hassan (Red Hat)
2:35pm3:15pm Tuesday, March 17, 2020
Location: LL20D

Who is this presentation for?

Data engineers, data architects, developers

Level

Intermediate

Description

As log data continues to grow, it’s beneficial to automate analyzing logs to find anomalies and errors quickly with ML. Log anomaly detection is at its core an unsupervised NLP ML problem that can be difficult to validate and tune in a production setting. However, if there were a system that could incorporate minimal user feedback during model training, you could offset some of these challenges.

Zak Hassan shares lessons learned from building a log anomaly detection system in production, giving specific emphasis to technical challenges, scalability, implementing a human-in-the-loop ML system, and integrating ETL with unsupervised ML to detect anomalies in application logs.

Prerequisite knowledge

  • A basic knowledge of ML

What you'll learn

  • Discover the challenges in running ML in production
  • Take a deep dive into use case and application of log anomaly detection
  • Understand the ML model lifecycle
Photo of Zak Hassan

Zak Hassan

Red Hat

Zak Hassan is a software engineer on the data analytics team working on data science and machine learning at Red Hat. Previously, Zak was a software consultant in the financial services and insurance industry, building end-to-end software solutions for clients.

Leave a Comment or Question

Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?

Join the conversation here (requires login)

Contact us

confreg@oreilly.com

For conference registration information and customer service

partners@oreilly.com

For more information on community discounts and trade opportunities with O’Reilly conferences

Become a sponsor

For information on exhibiting or sponsoring a conference

pr@oreilly.com

For media/analyst press inquires