Lightweight Spectrum Prediction Based on Knowledge Distillation

Abstract

To address the challenges of increasing complexity and larger number of training samples required for high-accuracy spectrum prediction, we propose a novel lightweight model, leveraging a temporal convolutional network (TCN) and knowledge distillation. First, the prediction accuracy of TCN is enhanced via a self-transfer method. Then, we design a two-branch network which can extract the spectrum features efficiently. By employing knowledge distillation, we transfer the knowledge from TCN to the two-branch network, resulting in improved accuracy for spectrum prediction of the lightweight network. Experimental results show that the proposed model can improve accuracy by 19.5% compared to the widely-used LSTM model with sufficient historical data and reduces 71.1% parameters to be trained. Furthermore, the prediction accuracy is improved by 17.9% compared to Gated Recurrent Units (GRU) in the scenarios with scarce historical data.

Keywords

• spectrum prediction
• knowledge distillation
• temporal convolutional network
• lightweight networks
• few-shot learning