TY - JOUR
T1 - Semantics-Preserving Reinforcement Learning Attack Against Graph Neural Networks for Malware Detection
AU - Zhang, Lan
AU - Liu, Peng
AU - Choi, Yoon Ho
AU - Chen, Ping
N1 - Funding Information:
This work was supported in part by ARO under Grant W911NF-13-1-0421 (MURI), and in part by NSF under Grants CNS-1814679 and CNS-2019340.
Publisher Copyright:
© 2004-2012 IEEE.
PY - 2023/3/1
Y1 - 2023/3/1
N2 - As an increasing number of deep-learning-based malware scanners have been proposed, the existing evasion techniques, including code obfuscation and polymorphic malware, are found to be less effective. In this work, we propose a reinforcement learning based semantics-preserving (i.e. functionality-preserving) attack against black-box GNNs (Graph Neural Networks) for malware detection. The key factor of adversarial malware generation via semantic Nops insertion is to select the appropriate semantic Nops and their corresponding basic blocks. The proposed attack uses reinforcement learning to automatically make these 'how to select' decisions. To evaluate the attack, we have trained two kinds of GNNs with three types (e.g., Backdoor, Trojan, and Virus) of Windows malware samples and various benign Windows programs. The evaluation results have shown that the proposed attack can achieve a significantly higher evasion rate than four baseline attacks, namely the binary diversification attack, the semantics-preserving random instruction insertion attack, the semantics-preserving accumulative instruction insertion attack, and the semantics-preserving gradient-based instruction insertion attack.
AB - As an increasing number of deep-learning-based malware scanners have been proposed, the existing evasion techniques, including code obfuscation and polymorphic malware, are found to be less effective. In this work, we propose a reinforcement learning based semantics-preserving (i.e. functionality-preserving) attack against black-box GNNs (Graph Neural Networks) for malware detection. The key factor of adversarial malware generation via semantic Nops insertion is to select the appropriate semantic Nops and their corresponding basic blocks. The proposed attack uses reinforcement learning to automatically make these 'how to select' decisions. To evaluate the attack, we have trained two kinds of GNNs with three types (e.g., Backdoor, Trojan, and Virus) of Windows malware samples and various benign Windows programs. The evaluation results have shown that the proposed attack can achieve a significantly higher evasion rate than four baseline attacks, namely the binary diversification attack, the semantics-preserving random instruction insertion attack, the semantics-preserving accumulative instruction insertion attack, and the semantics-preserving gradient-based instruction insertion attack.
UR - http://www.scopus.com/inward/record.url?scp=85150883008&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85150883008&partnerID=8YFLogxK
U2 - 10.1109/TDSC.2022.3153844
DO - 10.1109/TDSC.2022.3153844
M3 - Article
AN - SCOPUS:85150883008
SN - 1545-5971
VL - 20
SP - 1390
EP - 1402
JO - IEEE Transactions on Dependable and Secure Computing
JF - IEEE Transactions on Dependable and Secure Computing
IS - 2
ER -