A fast extraction method of broccoli phenotype based on machine vision and deep learning

Abstract

How to accurately obtain the area and freshness of broccoli head in the field condition is the key step to determine broccoli growth. However, the rapid segmentation and grading of broccoli ball remains difficult due to the low equipment development level. In this research, we combined an advanced computer vision technique with a deep learning architecture to allow the acquisition of real-time and accurate information about broccoli ball. By constructing a private image dataset with 442 of broccoli-ball images (acquired using a self-developed imaging system) under controlled conditions, a deep convolutional neural network named “Improved ResNet” was trained to extract the broccoli pixels from the background. The technical process of our method includes: (1) take the orthophoto images of broccoli head based on a near-ground image acquisition platform and establish the original data set; (2) preprocess the training images and input the model for segmentation; (3) use the PSOA and Otsu algorithm for fine segment based on color characteristics to obtain the freshness information. The experimental results demonstrated that the precision of the segmentation model is about 0.9 which is robust to the interference of soil reflectance fluctuation, canopy shadow, leaf occlusion and so on. Our experiments showed that a combination of improved ResNet and PSOA method got higher broccoli balls segmenting and grading precision. One major advantage of this approach is that dealing with only a few images, reducing the data volume and memory requirements for the image processing. All of the methods were evaluated using ground-truth data from three different varieties, which we also make available to the research community for subsequent algorithm development and result comparison. Compared with other 4 approaches, the evaluation results shows better performance regarding the segmentation and grading accuracy. The results of SSIM, precision, recall and F-measure by using Improved ResNet were about 0.911, 0.897, 0.908 and 0.907 respectively, which were 10%~15% higher than the traditional approaches. In addition, on the basis of the segmentation results, PSO-Otsu method was proved that it can be used to achieve a quickly analysis to the freshness of the ball, with the mean accuracy of 0.82. Overall, the proposed method is a high-throughput method to acquire multi-phenotype parameters of broccoli in field condition, which can support the research of broccoli field monitoring and traits tracking.

Keywords

• deep learning
• broccoli phenotype
• computer vision
• automatic grading
• field platform