TY - GEN
T1 - A Scalable Mixture Model Based Defense Against Data Poisoning Attacks on Classifiers
AU - Li, Xi
AU - Miller, David J.
AU - Xiang, Zhen
AU - Kesidis, George
N1 - Funding Information:
This research is supported in part by an AFOSR DDDAS grant and a Cisco Systems URP gift.
Publisher Copyright:
© 2020, Springer Nature Switzerland AG.
PY - 2020
Y1 - 2020
N2 - Classifiers, e.g., those based on Naive Bayes, a support vector machine, or even a neural network, are highly susceptible to a data-poisoning attack. The attack objective is to degrade classification accuracy by covertly embedding malicious (labeled) samples into the training set. Such attacks can be mounted by an insider, through an outsourcing process (for data acquisition or training), or conceivably during active learning. In some cases, a very small amount of poisoning can result in dramatic reduction in classification accuracy. Data poisoning attacks are successful mainly because the malicious injected samples significantly skew the data distribution of the corrupted class. Such attack samples are generally data outliers and in principle separable from the clean samples. We propose a generalized, scalable, and dynamic data driven defense system that: 1) uses a mixture model both to well-fit the (potentially multi-modal) data and to give potential to isolate attack samples in a small subset of the mixture components; 2) performs hypothesis testing to decide both which components and which samples within those components are poisoned, with the identified poisoned ones purged from the training set. Our approaches addresses the attack scenario where adversarial samples are an unknown subset embedded in the initial training set, and can be used to perform data sanitization as a precursor to the training of any type of classifier. The promising results for experiments on the TREC05 spam corpus and Amazon reviews polarity dataset demonstrate the effectiveness of our defense strategy.
AB - Classifiers, e.g., those based on Naive Bayes, a support vector machine, or even a neural network, are highly susceptible to a data-poisoning attack. The attack objective is to degrade classification accuracy by covertly embedding malicious (labeled) samples into the training set. Such attacks can be mounted by an insider, through an outsourcing process (for data acquisition or training), or conceivably during active learning. In some cases, a very small amount of poisoning can result in dramatic reduction in classification accuracy. Data poisoning attacks are successful mainly because the malicious injected samples significantly skew the data distribution of the corrupted class. Such attack samples are generally data outliers and in principle separable from the clean samples. We propose a generalized, scalable, and dynamic data driven defense system that: 1) uses a mixture model both to well-fit the (potentially multi-modal) data and to give potential to isolate attack samples in a small subset of the mixture components; 2) performs hypothesis testing to decide both which components and which samples within those components are poisoned, with the identified poisoned ones purged from the training set. Our approaches addresses the attack scenario where adversarial samples are an unknown subset embedded in the initial training set, and can be used to perform data sanitization as a precursor to the training of any type of classifier. The promising results for experiments on the TREC05 spam corpus and Amazon reviews polarity dataset demonstrate the effectiveness of our defense strategy.
UR - http://www.scopus.com/inward/record.url?scp=85097425457&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85097425457&partnerID=8YFLogxK
U2 - 10.1007/978-3-030-61725-7_31
DO - 10.1007/978-3-030-61725-7_31
M3 - Conference contribution
AN - SCOPUS:85097425457
SN - 9783030617240
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 262
EP - 273
BT - Dynamic Data Driven Application Systems - Third International Conference, DDDAS 2020, Proceedings
A2 - Darema, Frederica
A2 - Blasch, Erik
A2 - Ravela, Sai
A2 - Aved, Alex
PB - Springer Science and Business Media Deutschland GmbH
T2 - 3rd International Conference on Dynamic Data Driven Application Systems, DDDAS 2020
Y2 - 2 October 2020 through 4 October 2020
ER -