TCNet: Transformer Convolution Network for Cutting-Edge Detection of Unharvested Rice Regions

Abstract

Cutting-edge detection is a critical step in mechanized rice harvesting. Through visual cutting-edge detection, an algorithm can sense in real-time whether the rice harvesting process is along the cutting-edge, reducing loss and improving the efficiency of mechanized harvest. Although convolutional neural network-based models, which have strong local feature acquisition ability, have been widely used in rice production, these models involve large receptive fields only in the deep network. Besides, a self-attention-based Transformer can effectively provide global features to complement the disadvantages of CNNs. Hence, to quickly and accurately complete the task of cutting-edge detection in a complex rice harvesting environment, this article develops a Transformer Convolution Network (TCNet). This cutting-edge detection algorithm combines the Transformer with a CNN. Specifically, the Transformer realizes a patch embedding through a 3 × 3 convolution, and the output is employed as the input of the Transformer module. Additionally, the multi-head attention in the Transformer module undergoes dimensionality reduction to reduce overall network computation. In the Feed-forward network, a 7 × 7 convolution operation is used to realize the position-coding of different patches. Moreover, CNN uses depth-separable convolutions to extract local features from the images. The global features extracted by the Transformer and the local features extracted by the CNN are integrated into the fusion module. The test results demonstrated that TCNet could segment 97.88% of the Intersection over Union and 98.95% of the Accuracy in the unharvested region, and the number of parameters is only 10.796M. Cutting-edge detection is better than common lightweight backbone networks, achieving the detection effect of deep convolutional networks (ResNet-50) with fewer parameters. The proposed TCNet shows the advantages of a Transformer combined with a CNN and provides real-time and reliable reference information for the subsequent operation of rice harvesting.

Keywords

• rice harvesting
• cutting-edge
• semantic segmentation
• unharvested region
• feature fusion