A survey of efficient deep neural network

Abstract

Recently,deep neural network (DNN) has achieved great success in the field of AI such as computer vision and natural language processing.Thanks to a deeper and larger network structure,DNN’s performance is rapidly increasing.However,deeper and lager deep neural networks require huge computational and memory resources.In some resource-constrained scenarios,it is difficult to deploy large neural network models.How to design a lightweight and efficient deep neural network to accelerate its running speed on embedded devices is a great research hotspot for advancing deep neural network technology.The research methods and work of representative high-efficiency deep neural networks in recent years were reviewed and summarized,including parameter pruning,model quantification,knowledge distillation,network search and quantification.Also,vadvantages and disadvantages of different methods as well as applicable scenarios were analyzed,and the future development trend of efficient neural network design was forecasted.

Keywords

• deep neural network
• model accelerator and compression
• knowledge distillation