A Multimodal Data Fusion and Deep Neural Networks Based Technique for Tea Yield Estimation in Pakistan Using Satellite Imagery

Abstract

Achieving food security has become a major challenge for society. Crop yield estimation is essential for crop monitoring to ensure food security. Manual crop yield estimation is cumbersome and inaccurate and becomes infeasible when scaled up. Machine learning algorithms trained using remotely sensed data have played a vital role in estimating the yield of different crops. Furthermore, to enrich the data provided to a machine learning algorithm, multiple modalities can be combined to improve the predictive performance of these algorithms. In this research, we propose to combine data from multiple modalities, i.e., agrometeorological and remote sensing data, to predict the tea yield at the farm level. The dataset employed in this study is acquired from tea fields of the National Tea and High-Value Crop Research Institute (NTHRI), Mansehra, Pakistan. The remote sensing data of the Landsat-8 satellite is converted to farm-level NDVI statistics through geocoding. Before being used for regression modeling, the final dataset is subjected to some further preprocessing steps, including the selection of features and the optimization of feature sets. This preprocessed data is used to train the three classes of machine learning regression algorithms. Conventional regression algorithms, including Decision Trees, Multilayer Perceptron (MLP), Support Vector Regression (SVR), Gaussian Process Regression (GPR), and Multiple Linear Regression applied with and without interaction terms and stepwise feature inclusion with various kernels. Moreover, the following three variants of the ensemble learning methods have also been applied: random forest, gradient boosting, and XgBoost. Finally, this study proposed a neural architecture for tea yield estimation using Landsat imagery. This deep neural network is built using neural architecture search via Bayesian optimization and have three hidden layers, which can perform complex non-linear modeling. Experimental evaluation is performed through 10-fold cross-validation, and the proposed Deep neural network regression model provided the best predictive performance. The model provided a coefficient of determination (R-squared) of 0.99 with a Mean Square Error (MSE) of 108.17 kg/ha, Root Mean Square Error (RMSE) of 10.87 kg/ha, Mean Absolute Error (MAE) of 2.26 kg/ha and Mean Absolute Percentage Error (MAPE) of 2.92.

Keywords

• Multimodal
• data fusion
• deep neural networks
• crop yield forecasting tea yield
• Pakistan
• satellite imagery