Agri-LLaVA: Knowledge-Infused Large Multimodal Assistant on Agricultural Pests and Diseases

• URL: http://arxiv.org/abs/2412.02158v2
• Date: Wed, 04 Dec 2024 08:34:49 GMT
• Title: Agri-LLaVA: Knowledge-Infused Large Multimodal Assistant on Agricultural Pests and Diseases
• Authors: Liqiong Wang, Teng Jin, Jinyu Yang, Ales Leonardis, Fangyi Wang, Feng Zheng,
• Abstract summary: We construct the first multimodal instruction-following dataset in the agricultural domain. This dataset covers over 221 types of pests and diseases with approximately 400,000 data entries. We propose a knowledge-infused training method to develop Agri-LLaVA, an agricultural multimodal conversation system.
• Score: 49.782064512495495
• License:
• Abstract: In the general domain, large multimodal models (LMMs) have achieved significant advancements, yet challenges persist in applying them to specific fields, especially agriculture. As the backbone of the global economy, agriculture confronts numerous challenges, with pests and diseases being particularly concerning due to their complexity, variability, rapid spread, and high resistance. This paper specifically addresses these issues. We construct the first multimodal instruction-following dataset in the agricultural domain, covering over 221 types of pests and diseases with approximately 400,000 data entries. This dataset aims to explore and address the unique challenges in pest and disease control. Based on this dataset, we propose a knowledge-infused training method to develop Agri-LLaVA, an agricultural multimodal conversation system. To accelerate progress in this field and inspire more researchers to engage, we design a diverse and challenging evaluation benchmark for agricultural pests and diseases. Experimental results demonstrate that Agri-LLaVA excels in agricultural multimodal conversation and visual understanding, providing new insights and approaches to address agricultural pests and diseases. By open-sourcing our dataset and model, we aim to promote research and development in LMMs within the agricultural domain and make significant contributions to tackle the challenges of agricultural pests and diseases. All resources can be found at https://github.com/Kki2Eve/Agri-LLaVA.

Related papers

• AgroGPT: Efficient Agricultural Vision-Language Model with Expert Tuning [30.034193330398292]
  We propose an approach to construct instruction-tuning data that harnesses vision-only data for the agriculture domain. We utilize diverse agricultural datasets spanning multiple domains, curate class-specific information, and employ large language models (LLMs) to construct an expert-tuning set. We expert-tuned and created AgroGPT, an efficient LMM that can hold complex agriculture-related conversations and provide useful insights.
  arXiv  Detail & Related papers  (2024-10-10T22:38:26Z)
• Harnessing Large Vision and Language Models in Agriculture: A Review [3.6673562709926664]
  Large models can help farmers improve production efficiency and harvest by detecting a series of agricultural production tasks. After gaining a deeper understanding of multimodal large language models (MLLM), it can be recognized that problems such as agricultural image processing, agricultural question answering systems, and agricultural machine automation can all be solved by large models.
  arXiv  Detail & Related papers  (2024-07-29T03:47:54Z)
• Application of Machine Learning in Agriculture: Recent Trends and Future Research Avenues [6.0460261046732455]
  Food production is a vital global concern and the potential for an agritech revolution through artificial intelligence (AI) remains largely unexplored. This paper presents a comprehensive review focused on the application of machine learning (ML) in agriculture, aiming to explore its transformative potential in farming practices and efficiency enhancement.
  arXiv  Detail & Related papers  (2024-05-23T17:53:31Z)
• Generating Diverse Agricultural Data for Vision-Based Farming Applications [74.79409721178489]
  This model is capable of simulating distinct growth stages of plants, diverse soil conditions, and randomized field arrangements under varying lighting conditions. Our dataset includes 12,000 images with semantic labels, offering a comprehensive resource for computer vision tasks in precision agriculture.
  arXiv  Detail & Related papers  (2024-03-27T08:42:47Z)
• Leaf-Based Plant Disease Detection and Explainable AI [16.128084819516715]
  The agricultural sector plays an essential role in the economic growth of a country. Plant Disease is one of the significant factors affecting the agricultural sector. Researchers have explored many applications based on AI and Machine Learning techniques to detect plant diseases.
  arXiv  Detail & Related papers  (2023-12-17T03:40:12Z)
• Climate Change Impact on Agricultural Land Suitability: An Interpretable Machine Learning-Based Eurasia Case Study [94.07737890568644]
  As of 2021, approximately 828 million people worldwide are experiencing hunger and malnutrition. Climate change significantly impacts agricultural land suitability, potentially leading to severe food shortages. Our study focuses on Central Eurasia, a region burdened with economic and social challenges.
  arXiv  Detail & Related papers  (2023-10-24T15:15:28Z)
• Empowering Agrifood System with Artificial Intelligence: A Survey of the Progress, Challenges and Opportunities [86.89427012495457]
  We review how AI techniques can transform agrifood systems and contribute to the modern agrifood industry. We present a progress review of AI methods in agrifood systems, specifically in agriculture, animal husbandry, and fishery. We highlight potential challenges and promising research opportunities for transforming modern agrifood systems with AI.
  arXiv  Detail & Related papers  (2023-05-03T05:16:54Z)
• The 1st Agriculture-Vision Challenge: Methods and Results [144.57794061346974]
  The first Agriculture-Vision Challenge aims to encourage research in developing novel and effective algorithms for agricultural pattern recognition from aerial images. Around 57 participating teams from various countries compete to achieve state-of-the-art in aerial agriculture semantic segmentation. This paper provides a summary of notable methods and results in the challenge.
  arXiv  Detail & Related papers  (2020-04-21T05:02:31Z)
• Data Warehouse and Decision Support on Integrated Crop Big Data [0.0]
  We designed and implemented a continental level agricultural data warehouse (ADW) ADW is characterised by its (1) flexible schema; (2) data integration from real agricultural multi datasets; (3) data science and business intelligent support; (4) high performance; (5) high storage; (6) security; (7) governance and monitoring; (8) consistency, availability and partition tolerant; (9) cloud compatibility.
  arXiv  Detail & Related papers  (2020-03-10T00:10:22Z)
• Agriculture-Vision: A Large Aerial Image Database for Agricultural Pattern Analysis [110.30849704592592]
  We present Agriculture-Vision: a large-scale aerial farmland image dataset for semantic segmentation of agricultural patterns. Each image consists of RGB and Near-infrared (NIR) channels with resolution as high as 10 cm per pixel. We annotate nine types of field anomaly patterns that are most important to farmers.
  arXiv  Detail & Related papers  (2020-01-05T20:19:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.