Execution Anomaly Detection in Distributed Systems through Unstructured Log Analysis

By Qiang Fu, Jian-Guang Lou, Yi Wang, Jiang Li. 2009 Ninth IEEE International Conference on Data Mining, 2009

604 citations recorded by the source corpus.

Subfields: Computer Networks and Communications, Information Systems, Software

View Paper: https://www.semanticscholar.org/paper/12d4c92f0a3a70538ed609bf6f7b603e44d11abd

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Information Systems

In Software

Citations in the corpus

Structured Comparative Analysis of Systems Logs to Diagnose Performance Problems (2012, methodology): “[12] propose the use of a Finite State Automaton to learn the structure of a normal execution, and use it to detect anomalies in performance of new input log files.”
this paper (2009, neutral): “For example, cloud gaming services deliver content with a resolution of ≥ 1080p [3] and frame-rate of 60fps [61], while requiring a tail end-to-end delay of less than 100ms [43].”
A Survey on Automated Log Analysis for Reliability Engineering (2020, methodology): “ion anomaly. A sequence anomaly is raised when an expected child is missing for a parent node within the given time interval; a distribution anomaly is raised when an edge probability is violated. In [49], Fu et al. modeled the execution behaviors of each system module as a state transition graph by”
Finding Likely Errors with Bayesian Specifications (2017, methodology): “There is prior work on using learned models of executions in anomaly detection (Wasylkowski et al. 2007; Monperrus et al. 2010; Hangal & Lam 2002; Chilimbi & Ganapathy 2006; Baah et al. 2006; Gao et al. 2007; Fu et al. 2009; Wang et al. 2016a).”
Log-based Anomaly Detection Without Log Parsing (2021, methodology): “He et al [16] evaluated four widely used log parsers, including SLCT [57], IPLoM [26], LKE [58] and LogSig [20].”
DeepLog: Anomaly Detection and Diagnosis from System Logs through Deep Learning (2017, methodology): “…that could be satisfied by the majority of vectors, and then treats those vectors that do not satisfy these invariants as abnormal execution sessions. approach is shown to be more effective than an earlier work [11] which utilizes workflow automata. implementation is open-sourced by [17].”
Experience Report: System Log Analysis for Anomaly Detection (2016, methodology): “However, there are some other methods employing different models, such as frequent sequence mining [22], finite state machine [20], formal concept analysis [18], and information retrieval [30].”
Robust log-based anomaly detection on unstable log data (2019, methodology): “There have been a lot of studies on log-based anomaly detection[2, 4, 7, 11, 12, 14, 15, 20, 26, 28, 29, 33–35, 38, 41].”
Spell: Streaming Parsing of System Event Logs (2016, methodology): “When a new log entry ei arrives, we first compare it with all LCSseq’s in existing LCSObjects in LCSMap, then based on the results, either insert the line id i to the lineIds of an existing LCSObject, or compute a new LCSObject and insert it into LCSMap.”
Bayesian specification learning for finding API usage errors (2017, methodology): “There is prior work on using learned models of executions in anomaly detection [10, 19, 22, 23, 28, 38, 52, 54].”
Mining Invariants from Console Logs for System Problem Detection (2010, methodology): “It can achieve an accuracy of more than 95% [14].”
Log Clustering Based Problem Identification for Online Service Systems (2016, methodology): “To facilitate analysis, a common practice is to parse the log messages into constant strings and parameters [6, 11, 12, 24] and form abstract log messages.”