Using Fuzzy Mask R-CNN Model to Automatically Identify Tomato Ripeness

Abstract

Manual inspection and harvesting of ripening tomatoes is time consuming and labor intensive. Smart agriculture can emphasize the use of digital horticultural resources for farming and can increase farm sustainability; to that end, we proposed a fuzzy Mask R-CNN model to automatically identify the ripeness levels of cherry tomatoes. First, to annotate the images automatically, a fuzzy c-means model was used to maintain the spatial information of various foreground and background elements of the image. Then, a Hough transform method was applied to locate the specific geometric edge positions of the tomatoes. Each data point of the image space was annotated to a JavaScript Object Notation file. Second, annotated images were trained with Mask R-CNN to identify each tomato precisely. Finally, to prevent preharvest abscission of tomatoes, a hue-saturation-value color model and fuzzy inference rules were used to predict the ripeness of the tomatoes. A trigonometric function with Euclidian distance was calculated from the origin of calyx and stem to the bottom of the tomato to obtain the position of the pedicle head and dissect the fruit in a timely manner. For detection of 100 tomato images, Mask R-CNN achieved an accuracy of 98.00%. The ripeness classification of tomatoes achieved overall weighted precision and recall rates of 0.9614 and 0.9591, respectively. Thus, automatic tomato harvesting applications can empower farmers to make better decisions and enhance overall production efficiency and yield.

Keywords

• Automatic annotation
• detection of tomato ripeness
• fuzzy c-means
• Mask Region-based Convolutional Neural Network (Mask R-CNN)
• hue--saturation--value (HSV) color model