Research on a Real-Time, High-Precision End-to-End Sorting System for Fresh-Cut Flowers

Abstract

As the quality of life rises, the demand for flowers has increased significantly, leading to higher expectations for flower sorting system efficiency and speed. This paper presents a real-time, high-precision end-to-end method, which can complete three key tasks in the sorting system: flower localization, flower classification, and flower grading. In order to improve the challenging maturity detection, red–green–blue depth (RGBD) images were captured. The multi-task and multi-dimension-You Only Look Once (MTMD-YOLO) network was proposed to complete these three tasks in an end-to-end manner. The feature fusion was simplified to increase training speed, and the detection head and non-maximum suppression (NMS) were optimized for the dataset. This optimization allowed the loss function for the grading task to be added to train each task separately. The results showed that the use of RGBD and multi-task improved by 3.63% and 1.87% of mean average precision (mAP) on flower grading task, respectively. The final mAP of the flower classification and grading task reached 98.19% and 97.81%, respectively. The method also achieved real-time speed on embedded Jetson Orin NX, with 37 frames per second (FPS). This method provided essential technical support to determine the automatic flower picking times, in combination with a picking robot.

Keywords

• fresh-cut flower sorting system
• RGBD images
• multi-task and multi-dimension-You Only Look Once