Multiple flow‐based knowledge transfer via adversarial networks

Abstract

The authors propose a new knowledge transfer method coupled with a generative adversarial network (GAN) when multiple‐flow‐based knowledge is considered in a teacher–student framework using a residual network (ResNet). In this method, several independent discriminators adapting multilayer‐perceptron‐based structures were designed for flow‐based knowledge transfer. The proposed GAN‐based optimisation alternatively updates the multiple discriminators and a student ResNet such that the flow‐based features of the student ResNet are generated as closely as possible to the real features of a teacher ResNet. The experiments demonstrate that the student ResNet trained using the proposed method more accurately captures the distribution of the flow‐based teacher knowledge than the l2‐distance‐based training method. In addition, the proposed method provided better classification accuracy than the existing GAN‐based knowledge transfer method.

Keywords

• multiple flow‐based knowledge transfer
• generative adversarial network
• teacher–student framework
• residual network
• independent discriminators
• multilayer‐perceptron‐based structures