LogPAI

From logs to system insights

LogPAI brings together open datasets and software for automated log analysis. Its tools support the steps from structuring raw log messages to identifying unusual system behavior.

Network outage monitoring at IBM

IBM selected Drain for log-template mining and extended it into Drain3 for its production pipeline. The IBM case study describes using logs from IBM Cloud network devices to detect changes in message frequency and identify potential network incidents. The image above shows anomaly detection for one log template from a single network device.

Datasets, parsing, and anomaly detection

Loghub provides system log datasets for research. Logparser turns unstructured messages into structured events. Loglizer implements machine-learning methods for log-based anomaly detection and provides benchmarking examples.

Loglizer accompanies the ISSRE 2016 paper Experience Report: System Log Analysis for Anomaly Detection, listed below. The LogPAI website includes the broader publication history, and the GitHub organization provides access to the projects.

Research summary prepared from the public source for this prototype; the selection and editorial wording await owner review.

From logs to system insights

Network outage monitoring at IBM

Datasets, parsing, and anomaly detection

Related Publications

Drain: An Online Log Parsing Approach with Fixed Depth Tree

Experience Report: System Log Analysis for Anomaly Detection

Loghub: A Large Collection of System Log Datasets for AI-driven Log Analytics

Tools and Benchmarks for Automated Log Parsing