TY - GEN
T1 - Improving Neural Network Robustness Through Neighborhood Preserving Layers
AU - Liu, Bingyuan
AU - Malon, Christopher
AU - Xue, Lingzhou
AU - Kruus, Erik
N1 - Publisher Copyright:
© 2021, Springer Nature Switzerland AG.
PY - 2021
Y1 - 2021
N2 - One major source of vulnerability of neural nets in classification tasks is from overparameterized fully connected layers near the end of the network. In this paper, we propose a new neighborhood preserving layer which can replace these fully connected layers to improve the network robustness. Networks including these neighborhood preserving layers can be trained efficiently. We theoretically prove that our proposed layers are more robust against distortion because they effectively control the magnitude of gradients. Finally, we empirically show that networks with our proposed layers are more robust against state-of-the-art gradient descent based attacks, such as a PGD attack on the benchmark image classification datasets MNIST and CIFAR10.
AB - One major source of vulnerability of neural nets in classification tasks is from overparameterized fully connected layers near the end of the network. In this paper, we propose a new neighborhood preserving layer which can replace these fully connected layers to improve the network robustness. Networks including these neighborhood preserving layers can be trained efficiently. We theoretically prove that our proposed layers are more robust against distortion because they effectively control the magnitude of gradients. Finally, we empirically show that networks with our proposed layers are more robust against state-of-the-art gradient descent based attacks, such as a PGD attack on the benchmark image classification datasets MNIST and CIFAR10.
UR - http://www.scopus.com/inward/record.url?scp=85103238965&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85103238965&partnerID=8YFLogxK
U2 - 10.1007/978-3-030-68780-9_17
DO - 10.1007/978-3-030-68780-9_17
M3 - Conference contribution
AN - SCOPUS:85103238965
SN - 9783030687793
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 179
EP - 195
BT - Pattern Recognition. ICPR International Workshops and Challenges, 2021, Proceedings
A2 - Del Bimbo, Alberto
A2 - Cucchiara, Rita
A2 - Sclaroff, Stan
A2 - Farinella, Giovanni Maria
A2 - Mei, Tao
A2 - Bertini, Marco
A2 - Escalante, Hugo Jair
A2 - Vezzani, Roberto
PB - Springer Science and Business Media Deutschland GmbH
T2 - 25th International Conference on Pattern Recognition Workshops, ICPR 2020
Y2 - 10 January 2021 through 15 January 2021
ER -