GSP-AI: An AI-Powered Platform for Identifying Key Growth Stages and the Vegetative-to-Reproductive Transition in Wheat Using Trilateral Drone Imagery and Meteorological Data

Abstract

Wheat (Triticum aestivum) is one of the most important staple crops worldwide. To ensure its global supply, the timing and duration of its growth cycle needs to be closely monitored in the field so that necessary crop management activities can be arranged in a timely manner. Also, breeders and plant researchers need to evaluate growth stages (GSs) for tens of thousands of genotypes at the plot level, at different sites and across multiple seasons. These indicate the importance of providing a reliable and scalable toolkit to address the challenge so that the plot-level assessment of GS can be successfully conducted for different objectives in plant research. Here, we present a multimodal deep learning model called GSP-AI, capable of identifying key GSs and predicting the vegetative-to-reproductive transition (i.e., flowering days) in wheat based on drone-collected canopy images and multiseasonal climatic datasets. In the study, we first established an open Wheat Growth Stage Prediction (WGSP) dataset, consisting of 70,410 annotated images collected from 54 varieties cultivated in China, 109 in the United Kingdom, and 100 in the United States together with key climatic factors. Then, we built an effective learning architecture based on Res2Net and long short-term memory (LSTM) to learn canopy-level vision features and patterns of climatic changes between 2018 and 2021 growing seasons. Utilizing the model, we achieved an overall accuracy of 91.2% in identifying key GS and an average root mean square error (RMSE) of 5.6 d for forecasting the flowering days compared with manual scoring. We further tested and improved the GSP-AI model with high-resolution smartphone images collected in the 2021/2022 season in China, through which the accuracy of the model was enhanced to 93.4% for GS and RMSE reduced to 4.7 d for the flowering prediction. As a result, we believe that our work demonstrates a valuable advance to inform breeders and growers regarding the timing and duration of key plant growth and development phases at the plot level, facilitating them to conduct more effective crop selection and make agronomic decisions under complicated field conditions for wheat improvement.