Deep Learning Approach for Apple Edge Detection to Remotely Monitor Apple Growth in Orchards

Abstract

The automatic intelligent acquisition of apple growth information in the long-term provides a promising benefit for growers to plan the application of nutrients and pesticides during apple maturation. The overall goal of this study was to develop an apple growth monitoring system in an orchard based on a deep learning edge detection network for apple size remote estimation throughout the entire growth period. A remote apple growth monitoring hardware system was built with a spherical video camera and two personal computers to regularly acquire apple images. For software, an edge detection network that fused convolutional features (FCF) was proposed to segment the apple images. To filter out irrelevant apples in the images, points on apples to be monitored were manually selected from the images as seed points, and the region growing method was conducted on the extracted edge maps. Then, the horizontal diameters of the apples were calculated. The experimental results showed that the $F1$ score of the FCF method was 53.1% on the apple test set, and the average run time was 0.075 s per image, which was better than the other five methods in comparison. The growth of the apples was monitored by our system from the date after apple thinning to apple ripening. The mean average absolute error of the apples' horizontal diameters detected by our system was 0.90 mm, and it decreased by 67.9% when compared with the circle fitting-based method (2.8 mm). These results suggest that our system provides an effective and accurate way to monitor the growth of apples on the trees. The proposed method provides a reference for monitoring the growth of other fruits during the growth period, and it can be used to optimize orchard management.

Keywords

• Apple growth
• edge detection
• fruit horizontal diameter estimation
• remote monitoring
• ResNet-50