Multi-Temporal Data Fusion in MS and SAR Images Using the Dynamic Time Warping Method for Paddy Rice Classification

Abstract

This study employed a data fusion method to extract the high-similarity time series feature index of a dataset through the integration of MS (Multi-Spectrum) and SAR (Synthetic Aperture Radar) images. The farmlands are divided into small pieces that consider the different behaviors of farmers for their planting contents in Taiwan. Hence, the conventional image classification process cannot produce good outcomes. The crop phenological information will be a core factor to multi-period image data. Accordingly, the study intends to resolve the previous problem by using three different SPOT6 satellite images and nine Sentinel-1A synthetic aperture radar images, which were used to calculate features such as texture and indicator information, in 2019. Considering that a Dynamic Time Warping (DTW) index (i) can integrate different image data sources, (ii) can integrate data of different lengths, and (iii) can generate information with time characteristics, this type of index can resolve certain classification problems with long-term crop classification and monitoring. More specifically, this study used the time series data analysis of DTW to produce “multi-scale time series feature similarity indicators”. We used three approaches (Support Vector Machine, Neural Network, and Decision Tree) to classify paddy patches into two groups: (a) the first group did not apply a DTW index, and (b) the second group extracted conflict predicted data from (a) to apply a DTW index. The outcomes from the second group performed better than the first group in regard to overall accuracy (OA) and kappa. Among those classifiers, the Neural Network approach had the largest improvement of OA and kappa from 89.51, 0.66 to 92.63, 0.74, respectively. The rest of the two classifiers also showed progress. The best performance of classification results was obtained from the Decision Tree of 94.71, 0.81. Observing the outcomes, the interference effects of the image were resolved successfully by various image problems using the spectral image and radar image for paddy rice classification. The overall accuracy and kappa showed improvement, and the maximum kappa was enhanced by about 8%. The classification performance was improved by considering the DTW index.

Keywords

• remote sensing
• time series
• data fusion
• feature extraction
• dynamic time warping
• crop phenological information