Maui: modular analytics of UAS imagery for specialty crop research

Abstract

Abstract Background Imaging sensors (e.g., multispectral cameras) mounted on unmanned aerial systems (UAS) have emerged as a powerful tool for deriving insights about agricultural fields, from plant morphology phenotyping to plant disease monitoring. Advances in computer vision-based image analysis have enabled researchers to rapidly and accurately isolate crop spectra in UAS images. Specialty crops often employ unique production styles, such as trellising or inter-cropping. This presents a barrier to using existing image processing methodologies developed for broad-acre, row cropped systems (i.e. corn, wheat, soybean). Here, we present MAUI, a customizable image processing workflow built for specialty crops. Using a pathology research vineyard and hemp breeding trial as test cases, MAUI streamlines the generation of multispectral orthomosaic time-series, the segmentation of crops at the unit of research interest, and the extraction of crop spectra for downstream analysis. Results We successfully used MAUI to collect and analyze UAS data at two field sites over two growing seasons. Of the five canopy segmentation methods we tested, a supervised deep convolutional neural network (DeepLabv3) and a vision foundation model (SAM) produced the most accurate crop masks for the vineyard and hemp images, with mean intersection over union (mIoU) values of 0.85 and 0.95, respectively. Segmentation accuracy decreased when we applied each method to the other dataset, highlighting the importance of modular, flexible segmentation workflows for UAS imaging analysis in specialty crops. Conclusion We present a modular framework to efficiently extract spectral data for specialty crops from UAS imagery. We highlight two kinds of segmentation applied to trellised and row cropping systems to demonstrate the modularity and versatility of the proposed methodology. MAUI improved spectral discrimination between individual plants and treatment groups for hemp and grapevine, respectively. With the containerized deployment package and open-source codebase, MAUI can be widely adopted by specialty crop researchers to facilitate the integration of UAS imagery analysis into routine research.

Keywords

• UAS
• Specialty crops
• Hemp
• Grapevine
• Image segmentation
• Vision foundation model