High-Accuracy Tomato Leaf Disease Image-Text Retrieval Method Utilizing LAFANet

Jiaxin Xu; Hongliang Zhou; Yufan Hu; Yongfei Xue; Guoxiong Zhou; Liujun Li; Weisi Dai; Jinyang Li

doi:10.3390/plants13091176

Plants (Apr 2024)

High-Accuracy Tomato Leaf Disease Image-Text Retrieval Method Utilizing LAFANet

Jiaxin Xu,
Hongliang Zhou,
Yufan Hu,
Yongfei Xue,
Guoxiong Zhou,
Liujun Li,
Weisi Dai,
Jinyang Li

Affiliations

Jiaxin Xu: College of Computer and Information Engineering, Central South University of Forestry and Technology, Changsha 410004, China
Hongliang Zhou: College of Computer and Information Engineering, Central South University of Forestry and Technology, Changsha 410004, China
Yufan Hu: College of Computer and Information Engineering, Central South University of Forestry and Technology, Changsha 410004, China
Yongfei Xue: College of Computer and Information Engineering, Central South University of Forestry and Technology, Changsha 410004, China
Guoxiong Zhou: College of Computer and Information Engineering, Central South University of Forestry and Technology, Changsha 410004, China
Liujun Li: Department of Soil and Water Systems, University of Idaho, Moscow, ID 83844, USA
Weisi Dai: College of Computer and Information Engineering, Central South University of Forestry and Technology, Changsha 410004, China
Jinyang Li: College of Computer and Information Engineering, Central South University of Forestry and Technology, Changsha 410004, China

DOI: https://doi.org/10.3390/plants13091176
Journal volume & issue: Vol. 13, no. 9
p. 1176

Abstract

Read online

Tomato leaf disease control in the field of smart agriculture urgently requires attention and reinforcement. This paper proposes a method called LAFANet for image-text retrieval, which integrates image and text information for joint analysis of multimodal data, helping agricultural practitioners to provide more comprehensive and in-depth diagnostic evidence to ensure the quality and yield of tomatoes. First, we focus on six common tomato leaf disease images and text descriptions, creating a Tomato Leaf Disease Image-Text Retrieval Dataset (TLDITRD), introducing image-text retrieval into the field of tomato leaf disease retrieval. Then, utilizing ViT and BERT models, we extract detailed image features and sequences of textual features, incorporating contextual information from image-text pairs. To address errors in image-text retrieval caused by complex backgrounds, we propose Learnable Fusion Attention (LFA) to amplify the fusion of textual and image features, thereby extracting substantial semantic insights from both modalities. To delve further into the semantic connections across various modalities, we propose a False Negative Elimination-Adversarial Negative Selection (FNE-ANS) approach. This method aims to identify adversarial negative instances that specifically target false negatives within the triplet function, thereby imposing constraints on the model. To bolster the model’s capacity for generalization and precision, we propose Adversarial Regularization (AR). This approach involves incorporating adversarial perturbations during model training, thereby fortifying its resilience and adaptability to slight variations in input data. Experimental results show that, compared with existing ultramodern models, LAFANet outperformed existing models on TLDITRD dataset, with top1, top5, and top10 reaching 83.3% and 90.0%, and top1, top5, and top10 reaching 80.3%, 93.7%, and 96.3%. LAFANet offers fresh technical backing and algorithmic insights for the retrieval of tomato leaf disease through image-text correlation.

Published in Plants

ISSN: 2223-7747 (Online)
Publisher: MDPI AG
Country of publisher: Switzerland
LCC subjects: Science: Botany
Website: http://www.mdpi.com/journal/plants

About the journal

Abstract

Keywords