Log anomaly detector with NLP and unsupervised machine learning
Who is this presentation for?Data engineers, data architects, developers
As log data continues to grow, it’s beneficial to automate analyzing logs to find anomalies and errors quickly with ML. Log anomaly detection is at its core an unsupervised NLP ML problem that can be difficult to validate and tune in a production setting. However, if there were a system that could incorporate minimal user feedback during model training, you could offset some of these challenges.
Zak Hassan shares lessons learned from building a log anomaly detection system in production, giving specific emphasis to technical challenges, scalability, implementing a human-in-the-loop ML system, and integrating ETL with unsupervised ML to detect anomalies in application logs.
- A basic knowledge of ML
What you'll learn
- Discover the challenges in running ML in production
- Take a deep dive into use case and application of log anomaly detection
- Understand the ML model lifecycle
Zak Hassan is a software engineer on the data analytics team working on data science and machine learning at Red Hat. Previously, Zak was a software consultant in the financial services and insurance industry, building end-to-end software solutions for clients.
Leave a Comment or Question
Help us make this conference the best it can be for you. Have questions you'd like this speaker to address? Suggestions for issues that deserve extra attention? Feedback that you'd like to share with the speaker and other attendees?
Join the conversation here (requires login)
Premier Diamond Sponsors
Premier Exhibitor Plus
For conference registration information and customer service
For more information on community discounts and trade opportunities with O’Reilly conferences
For information on exhibiting or sponsoring a conference
For media/analyst press inquires