Improved learning by using a modified activation function of a Convolutional Neural Network in multi-spectral image classification

Abstract

The Convolutional Neural Network (CNN) algorithm is used to classify multispectral images of labelled EuroSAT data from Sentinel-2 satellite. The main objective of this study to examine the role of newly proposed activation function, Modified Rectified Linear Unit (MReLU) in improving multi-spectral image classification accuracy. The experimental design deployed a 5-layer CNN with activation function Rectified Linear Unit (ReLU) and the MReLU having 27 learning rates ranged from 0.0001 to 0.01 with 10 % increment and 8 optimizers. ReLU has a diminishing gradient problem since it does not consider the significant information present in the negative values of the convolution layer for different spectral bands. In order to overcome this short coming, we have designed the MReLU which allows the absolute values of the small negative gradient and thereby introducing higher nonlinearity in the output. The experimental results show that the image classification accuracy of the CNN with ReLU and MReLU activation functions is definitely higher than the pre-trained ResNet-50 model which is a complex and computationally expensive architecture. The MReLU has advantage over ReLU in land use classification, particularly in the Industrial, Herbaceous Vegetation, and Permanent Crop classes where accuracies were typically low. In summary, this paper highlights the importance of deploying a suitable activation function and hyperparameter tuning in the CNN model for improving the accuracy of multispectral image classification.

Keywords

• Convolutional Neural Network
• Activation function
• Rectified linear unit
• Land use and land cover
• Deep learning
• Multispectral images