Adaptive Cyber Defense against Multi-Stage Attacks Using Learning-Based POMDP

Zhisheng Hu, Minghui Zhu, Peng Liu

Research output: Contribution to journalArticlepeer-review

13 Scopus citations


Growing multi-stage attacks in computer networks impose significant security risks and necessitate the development of effective defense schemes that are able to autonomously respond to intrusions during vulnerability windows. However, the defender faces several real-world challenges, e.g., unknown likelihoods and unknown impacts of successful exploits. In this article, we leverage reinforcement learning to develop an innovative adaptive cyber defense to maximize the cost-effectiveness subject to the aforementioned challenges. In particular, we use Bayesian attack graphs to model the interactions between the attacker and networks. Then we formulate the defense problem of interest as a partially observable Markov decision process problem where the defender maintains belief states to estimate system states, leverages Thompson sampling to estimate transition probabilities, and utilizes reinforcement learning to choose optimal defense actions using measured utility values. The algorithm performance is verified via numerical simulations based on real-world attacks.

Original languageEnglish (US)
Article number6
JournalACM Transactions on Privacy and Security
Issue number1
StatePublished - Nov 2020

All Science Journal Classification (ASJC) codes

  • General Computer Science
  • Safety, Risk, Reliability and Quality


Dive into the research topics of 'Adaptive Cyber Defense against Multi-Stage Attacks Using Learning-Based POMDP'. Together they form a unique fingerprint.

Cite this