- Title
- Semi-supervised and unsupervised anomaly detection by mining numerical workflow relations from system logs
- Creator
- Zhang, Bo; Zhang, Hongyu; Le, Van-Hoang; Moscato, Pablo; Zhang, Aozhong
- Relation
- ARC.DP200102940 http://purl.org/au-research/grants/arc/DP200102940
- Relation
- Automated Software Engineering Vol. 30, Issue 1, no. 4
- Publisher Link
- http://dx.doi.org/10.1007/s10515-022-00370-w
- Publisher
- Springer
- Resource Type
- journal article
- Date
- 2023
- Description
- Large-scale software-intensive systems often generate logs for troubleshooting purpose. The system logs are semi-structured text messages that record the internal status of a system at runtime. In this paper, we propose ADR (Anomaly Detection by workflow Relations), which can mine numerical relations from logs and then utilize the discovered relations to detect system anomalies. Firstly the raw log entries are parsed into sequences of log events and transformed to an extended event-count-matrix. The relations among the matrix columns represent the relations among the system events in workflows. Next, ADR evaluates the matrix’s nullspace that corresponds to the linearly dependent relations of the columns. Anomalies can be detected by evaluating whether or not the logs violate the mined relations. We design two types of ADR: sADR (for semi-supervised learning) and uADR (for unsupervised learning). We have evaluated them on four public log datasets. The experimental results show that ADR can extract the workflow relations from log data, and is effective for log-based anomaly detection in both semi-supervised and unsupervised manners.
- Subject
- logs; anomaly detection; numerical relations; log analysis
- Identifier
- http://hdl.handle.net/1959.13/1477945
- Identifier
- uon:50063
- Identifier
- ISSN:0928-8910
- Language
- eng
- Reviewed
- Hits: 2038
- Visitors: 2037
- Downloads: 0
| Thumbnail | File | Description | Size | Format |
|---|