Panicle-3D: A low-cost 3D-modeling method for rice panicles based on deep learning, shape from silhouette, and supervoxel clustering
Dan Wu,
Lejun Yu,
Junli Ye,
Ruifang Zhai,
Lingfeng Duan,
Lingbo Liu,
Nai Wu,
Zedong Geng,
Jingbo Fu,
Chenglong Huang,
Shangbin Chen,
Qian Liu,
Wanneng Yang
Affiliations
Dan Wu
Britton Chance Center for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics, and Key Laboratory of Ministry of Education for Biomedical Photonics, Department of Biomedical Engineering, Huazhong University of Science and Technology, Wuhan 430074, Hubei, China
Lejun Yu
Britton Chance Center for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics, and Key Laboratory of Ministry of Education for Biomedical Photonics, Department of Biomedical Engineering, Huazhong University of Science and Technology, Wuhan 430074, Hubei, China; School of Biomedical Engineering, Hainan University, Haikou 570228, Hainan, China
Junli Ye
National Key Laboratory of Crop Genetic Improvement, National Center of Plant Gene Research, Huazhong Agricultural University, Wuhan 430070, Hubei, China
Ruifang Zhai
National Key Laboratory of Crop Genetic Improvement, National Center of Plant Gene Research, Huazhong Agricultural University, Wuhan 430070, Hubei, China
Lingfeng Duan
National Key Laboratory of Crop Genetic Improvement, National Center of Plant Gene Research, Huazhong Agricultural University, Wuhan 430070, Hubei, China
Lingbo Liu
Britton Chance Center for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics, and Key Laboratory of Ministry of Education for Biomedical Photonics, Department of Biomedical Engineering, Huazhong University of Science and Technology, Wuhan 430074, Hubei, China
Nai Wu
National Key Laboratory of Crop Genetic Improvement, National Center of Plant Gene Research, Huazhong Agricultural University, Wuhan 430070, Hubei, China
Zedong Geng
National Key Laboratory of Crop Genetic Improvement, National Center of Plant Gene Research, Huazhong Agricultural University, Wuhan 430070, Hubei, China
Jingbo Fu
National Key Laboratory of Crop Genetic Improvement, National Center of Plant Gene Research, Huazhong Agricultural University, Wuhan 430070, Hubei, China
Chenglong Huang
National Key Laboratory of Crop Genetic Improvement, National Center of Plant Gene Research, Huazhong Agricultural University, Wuhan 430070, Hubei, China
Shangbin Chen
Britton Chance Center for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics, and Key Laboratory of Ministry of Education for Biomedical Photonics, Department of Biomedical Engineering, Huazhong University of Science and Technology, Wuhan 430074, Hubei, China
Qian Liu
Britton Chance Center for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics, and Key Laboratory of Ministry of Education for Biomedical Photonics, Department of Biomedical Engineering, Huazhong University of Science and Technology, Wuhan 430074, Hubei, China; School of Biomedical Engineering, Hainan University, Haikou 570228, Hainan, China; Corresponding authors.
Wanneng Yang
National Key Laboratory of Crop Genetic Improvement, National Center of Plant Gene Research, Huazhong Agricultural University, Wuhan 430070, Hubei, China; Corresponding authors.
Self-occlusions are common in rice canopy images and strongly influence the calculation accuracies of panicle traits. Such interference can be largely eliminated if panicles are phenotyped at the 3D level. Research on 3D panicle phenotyping has been limited. Given that existing 3D modeling techniques do not focus on specified parts of a target object, an efficient method for panicle modeling of large numbers of rice plants is lacking. This paper presents an automatic and nondestructive method for 3D panicle modeling. The proposed method integrates shoot rice reconstruction with shape from silhouette, 2D panicle segmentation with a deep convolutional neural network, and 3D panicle segmentation with ray tracing and supervoxel clustering. A multiview imaging system was built to acquire image sequences of rice canopies with an efficiency of approximately 4 min per rice plant. The execution time of panicle modeling per rice plant using 90 images was approximately 26 min. The outputs of the algorithm for a single rice plant are a shoot rice model, surface shoot rice model, panicle model, and surface panicle model, all represented by a list of spatial coordinates. The efficiency and performance were evaluated and compared with the classical structure-from-motion algorithm. The results demonstrated that the proposed method is well qualified to recover the 3D shapes of rice panicles from multiview images and is readily adaptable to rice plants of diverse accessions and growth stages. The proposed algorithm is superior to the structure-from-motion method in terms of texture preservation and computational efficiency. The sample images and implementation of the algorithm are available online. This automatic, cost-efficient, and nondestructive method of 3D panicle modeling may be applied to high-throughput 3D phenotyping of large rice populations.
