Robust CNN Compression Framework for Security-Sensitive Embedded Systems

• Main Authors: Jeonghyun Lee, Sangkyun Lee
• Format: Article
• Language: English
• Published: MDPI AG 2021-01-01
• Series: Applied Sciences
• Subjects: model compression; adversarial robustness; weight pruning; adversarial training; distillation; embedded system
• Online Access: https://www.mdpi.com/2076-3417/11/3/1093

Convolutional neural networks (CNNs) have achieved tremendous success in solving complex classification problems. Motivated by this success, there have been proposed various compression methods for downsizing the CNNs to deploy them on resource-constrained embedded systems. However, a new type of vulnerability of compressed CNNs known as the adversarial examples has been discovered recently, which is critical for security-sensitive systems because the adversarial examples can cause malfunction of CNNs and can be crafted easily in many cases. In this paper, we proposed a compression framework to produce compressed CNNs robust against such adversarial examples. To achieve the goal, our framework uses both pruning and knowledge distillation with adversarial training. We formulate our framework as an optimization problem and provide a solution algorithm based on the proximal gradient method, which is more memory-efficient than the popular ADMM-based compression approaches. In experiments, we show that our framework can improve the trade-off between adversarial robustness and compression rate compared to the existing state-of-the-art adversarial pruning approach.