Patronus: Plug-and-Play and Near-Lossless Facial Privacy Enhancement Against Reconstruction Attacks

Reconstruction attackers can exploit facial features to recover the original user’s face, resulting in user privacy leakage. One new strategy to enhance the “Edge-Cloud” face recognition system’s privacy is to add adversarial perturbations to facial features, preventing the attackers from high-quality user image recovery. However, the existing works following this strategy suffer from unacceptable damage to face recognition accuracy. Achieving robust privacy enhancement and face recognition accuracy simultaneously is still challenging. To tackle this challenge, we propose an adversarial perturbation-based plug-and-play privacy-enhancing method (Patronus) with robustness against face image reconstruction attacks and near-lossless face recognition performance. The key insight is derived from our observation that the feature distance between two face images of the same person is significantly lower than the threshold set in the face recognition system. This leaves room for adding adversarial perturbations to the facial features without compromising face recognition accuracy. Our strategy limits the amount of adversarial perturbations in a fine-grained manner to ensure that they are within the range of not damaging face recognition accuracy. Our evaluation shows the superior performance of Patronus in robustness against reconstruction attacks and near-lossless face recognition accuracy compared to state-of-the-art (SOTA) methods. Patronus can be easily integrated into deployed face recognition systems as a plug-in privacy-enhancing module with low overhead.