Novel defense based on softmax activation transformation

Abstract

Deep learning is widely used in various fields such as image processing, natural language processing, network mining and so on.However, it is vulnerable to malicious adversarial attacks and many defensive methods have been proposed accordingly.Most defense methods are attack-dependent and require defenders to generate massive adversarial examples in advance.The defense cost is high and it is difficult to resist black-box attacks.Some of these defenses even affect the recognition of normal examples.In addition, the current defense methods are mostly empirical, without certifiable theoretical support.Softmax activation transformation (SAT) was proposed in this paper, which was a light-weight and fast defense scheme against black-box attacks.SAT reactivates the output probability of the target model in the testing phase, and then it guarantees privacy of the probability information.As an attack-free defense, SAT not only avoids the burden of generating massive adversarial examples, but also realizes the advance defense of attacks.The activation of SAT is monotonic, so it will not affect the recognition of normal examples.During the activation process, a variable privacy protection transformation coefficient was designed to achieve dynamic defense.Above all, SAT is a certifiable defense that can derive the effectiveness and reliability of its defense based on softmax activation transformation.To evaluate the effectiveness of SAT, defense experiments against 9 attacks on MNIST, CIFAR10 and ImageNet datasets were conducted, and the average attack success rate was reduced from 87.06% to 5.94%.

Keywords

• deep learning;adversarial defense;certifiable;attack-free