DeepSyslog: Deep Anomaly Detection on Syslog Using Sentence Embedding and Metadata

Junwei Zhou, Yijia Qian, Qingtian Zou, Peng Liu, Jianwen Xiang

Research output: Contribution to journalArticlepeer-review

9 Scopus citations

Abstract

Anomaly events indicating the unhealthy status of the computer system are recorded in the system log (Syslog). Therefore, Syslog-based anomaly event detection is crucial for diagnosing system issues and problems. However, existing log-based anomaly detection approaches use raw and unstructured log entries <italic>independently</italic> and <italic>incompletely</italic>, i.e., without considering the context of each event and event metadata in the logs. They employ incomplete representation of unstructured log data, limiting the deep learning model&#x2019;s capacity in the early stage, which tends to omit anomaly events and cause false alarms. In this work, we propose DeepSyslog, which represents Syslog with the context of log events and event metadata in the logs. Inspired by the sequence nature of the log stream, we employ unsupervised sentence embedding to extract the semantic and context information hidden in the log stream, rather than word embedding or one-hot embedding, which only capture the similarities between log words. The sentence embedding is further integrated with event metadata to form complete representations of Syslog, which can distinguish the anomaly caused by the correlated log entries and exceptional event metadata in the log. The simulation results on widely used log datasets show that DeepSyslog achieves high performance compared with the existing log-based anomaly event detection approaches.

Original languageEnglish (US)
Pages (from-to)1
Number of pages1
JournalIEEE Transactions on Information Forensics and Security
DOIs
StatePublished - 2022

All Science Journal Classification (ASJC) codes

  • Safety, Risk, Reliability and Quality
  • Computer Networks and Communications

Fingerprint

Dive into the research topics of 'DeepSyslog: Deep Anomaly Detection on Syslog Using Sentence Embedding and Metadata'. Together they form a unique fingerprint.

Cite this