Modification of SqueezeNet for Devices with Limited Computational Resources

Abstract

In recent years, the computational approach has shifted from a statistical basis to deep neural network architectures which process the input without explicit knowledge that underlies the model. Many models with high accuracy have been proposed by training the datasets using high performance computing devices. However, only a few studies have examined its use on non-high-performance computers. In fact, most users, who are mostly researchers in certain fields (medical, geography, economics, etc.) sometimes need computers with limited computational resources to process datasets, from notebooks, personal computers, to mobile processor-based devices. This study proposes a basic model with good accuracy and can run lightly on the average computer so that it remains lightweight when used as a basis for advanced deep neural networks models, e.g., U-Net, SegNet, PSPNet, DeepLab, etc. Using several well-known basic methods as a baseline (SqueezeNet, ShuffleNet, GoogleNet, MobileNetV2, and ResNet), a model combining SqueezeNet with ResNet, termed Res-SqueezeNet, was formed. Testing results show that the proposed method has accuracy and inference time of 84.59% and 8.46 second, respectively, which has an accuracy of 2% higher than the SqueezeNet (82.53%) and is close to the accuracy of other baseline methods (from 84.93% to 0.88.01%) while still maintaining the inference speed (below nine second). In addition, residual part of the proposed method can be used to avoid vanishing gradient, hence, it can be implemented to solve more advanced problems which need a lot of layers, e.g., semantic segmentation, time-series prediction, etc.

Keywords

• deep learning, squeezenet, resnet, imagenet, convolutional layer