Snap and Diagnose: An Advanced Multimodal Retrieval System for Identifying Plant Diseases in the Wild

• URL: http://arxiv.org/abs/2408.14723v1
• Date: Tue, 27 Aug 2024 01:23:49 GMT
• Title: Snap and Diagnose: An Advanced Multimodal Retrieval System for Identifying Plant Diseases in the Wild
• Authors: Tianqi Wei, Zhi Chen, Xin Yu,
• Abstract summary: We develop a multimodal plant disease image retrieval system to support disease search based on either image or text prompts. We utilize the largest in-the-wild plant disease dataset PlantWild, which includes over 18,000 images across 89 categories. Cross-modal retrieval is achieved in the developed system, facilitated by a novel CLIP-based vision-language model.
• Score: 22.940441995788063
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Plant disease recognition is a critical task that ensures crop health and mitigates the damage caused by diseases. A handy tool that enables farmers to receive a diagnosis based on query pictures or the text description of suspicious plants is in high demand for initiating treatment before potential diseases spread further. In this paper, we develop a multimodal plant disease image retrieval system to support disease search based on either image or text prompts. Specifically, we utilize the largest in-the-wild plant disease dataset PlantWild, which includes over 18,000 images across 89 categories, to provide a comprehensive view of potential diseases relating to the query. Furthermore, cross-modal retrieval is achieved in the developed system, facilitated by a novel CLIP-based vision-language model that encodes both disease descriptions and disease images into the same latent space. Built on top of the retriever, our retrieval system allows users to upload either plant disease images or disease descriptions to retrieve the corresponding images with similar characteristics from the disease dataset to suggest candidate diseases for end users' consideration.

Related papers

• FairSkin: Fair Diffusion for Skin Disease Image Generation [54.29840149709033]
  Diffusion Model (DM) has become a leading method in generating synthetic medical images, but it suffers from a critical twofold bias. We propose FairSkin, a novel DM framework that mitigates these biases through a three-level resampling mechanism. Our approach significantly improves the diversity and quality of generated images, contributing to more equitable skin disease detection in clinical settings.
  arXiv  Detail & Related papers  (2024-10-29T21:37:03Z)
• Decomposing Disease Descriptions for Enhanced Pathology Detection: A Multi-Aspect Vision-Language Pre-training Framework [43.453943987647015]
  Medical vision language pre-training has emerged as a frontier of research, enabling zero-shot pathological recognition. Due to the complex semantics of biomedical texts, current methods struggle to align medical images with key pathological findings in unstructured reports. This is achieved by consulting a large language model and medical experts. Ours improves the accuracy of recent methods by up to 8.56% and 17.26% for seen and unseen categories, respectively.
  arXiv  Detail & Related papers  (2024-03-12T13:18:22Z)
• Detection of healthy and diseased crops in drone captured images using Deep Learning [0.0]
  Disruptions in the plant's normal state, caused by diseases, often interfere with essential plant activities. We propose a deep learning-based approach for efficient detection of plant diseases using drone-captured imagery.
  arXiv  Detail & Related papers  (2023-05-22T21:15:12Z)
• Image Quality Assessment for Foliar Disease Identification (AgroPath) [2.9718407579521675]
  Crop diseases are a major threat to food security and their rapid identification is important to prevent yield loss. Recent advances in computer vision and increasing penetration of smartphones have paved the way for smartphone-assisted disease identification. This study was conducted in 2020 at Department of Computer Science and Engineering, University of Engineering and Technology, Lahore, Pakistan to check leaf-based plant disease identification.
  arXiv  Detail & Related papers  (2022-09-26T06:20:35Z)
• Paddy Doctor: A Visual Image Dataset for Paddy Disease Classification [0.0]
  Paddy Doctor is a visual image dataset for identifying paddy diseases. Our dataset contains 13,876 annotated paddy leaf images across ten classes (nine diseases and normal leaf) We benchmarked the Paddy Doctor using a Convolutional Neural Network (CNN) and two transfer learning approaches, VGG16 and MobileNet.
  arXiv  Detail & Related papers  (2022-05-23T07:57:40Z)
• Unsupervised deep learning techniques for powdery mildew recognition based on multispectral imaging [63.62764375279861]
  This paper presents a deep learning approach to automatically recognize powdery mildew on cucumber leaves. We focus on unsupervised deep learning techniques applied to multispectral imaging data. We propose the use of autoencoder architectures to investigate two strategies for disease detection.
  arXiv  Detail & Related papers  (2021-12-20T13:29:13Z)
• Generative Residual Attention Network for Disease Detection [51.60842580044539]
  We present a novel approach for disease generation in X-rays using a conditional generative adversarial learning. We generate a corresponding radiology image in a target domain while preserving the identity of the patient. We then use the generated X-ray image in the target domain to augment our training to improve the detection performance.
  arXiv  Detail & Related papers  (2021-10-25T14:15:57Z)
• An Interpretable Multiple-Instance Approach for the Detection of referable Diabetic Retinopathy from Fundus Images [72.94446225783697]
  We propose a machine learning system for the detection of referable Diabetic Retinopathy in fundus images. By extracting local information from image patches and combining it efficiently through an attention mechanism, our system is able to achieve high classification accuracy. We evaluate our approach on publicly available retinal image datasets, in which it exhibits near state-of-the-art performance.
  arXiv  Detail & Related papers  (2021-03-02T13:14:15Z)
• Image-based Plant Disease Diagnosis with Unsupervised Anomaly Detection Based on Reconstructability of Colors [0.0]
  We propose an unsupervised anomaly detection technique for image-based plant disease diagnosis. Our proposed method includes a new image-based framework for plant disease detection that utilizes a conditional adversarial network called pix2pix. Experiments with PlantVillage dataset demonstrated the superiority of our proposed method compared to an existing anomaly detector.
  arXiv  Detail & Related papers  (2020-11-29T07:44:05Z)
• A Multi-Plant Disease Diagnosis Method using Convolutional Neural Network [0.0]
  This chapter investigates an optimal plant disease identification model combining the diagnosis of multiple plants. We implement numerous popular convolutional neural network (CNN) architectures. The experimental results validate that the Xception and DenseNet architectures perform better in multi-label plant disease classification tasks.
  arXiv  Detail & Related papers  (2020-11-10T15:18:52Z)
• Multi-label Thoracic Disease Image Classification with Cross-Attention Networks [65.37531731899837]
  We propose a novel scheme of Cross-Attention Networks (CAN) for automated thoracic disease classification from chest x-ray images. We also design a new loss function that beyond cross-entropy loss to help cross-attention process and is able to overcome the imbalance between classes and easy-dominated samples within each class.
  arXiv  Detail & Related papers  (2020-07-21T14:37:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.