Field-grown tomato yield estimation using point cloud segmentation with 3D shaping and RGB pictures from a field robot and digital single lens reflex cameras

Abstract

The aim of this study was to estimate field-grown tomato yield (weight) and quantity of tomatoes using a self-developed robot and digital single lens reflex (DSLR) camera pictures. The authors suggest a new approach to predicting tomato yield that is based on images taken in the field, 3D scanning, and shape. Field pictures were used for tomato segmentation to determine the ripeness of the crop. A convolution neural network (CNN) model using TensorFlow library was devised for the segmentation of tomato berries along with a small robot, which had a 59.3 % F1 score. To enhance the accurate tomato crop model and to estimate the yield later, point cloud imaging was applied using a Ciclops 3D scanner. The best fitting sphere model was generated using the 3D model. The most optimal model was the 3D model, which gave the best representation and provided the weight of the tomatoes with a relative error of 21.90 % and a standard deviation of 17.9665 %. The results indicate a consistent object-based classification of the tomato crop above the plant/row level with an accuracy of 55.33 %, which is better than in-row sampling (images taken by the robot). By comparing the measured and estimated yield, the average difference for DSLR camera images was more favorable at 3.42 kg.

Keywords

• Tomato yield estimation
• Machine learning method
• Convolution neural network
• 3D point cloud shaping
• Approximation with sphere and 3D model
• Image segmentation and calibration