AgroGPT: Efficient Agricultural Vision-Language Model with Expert Tuning

• URL: http://arxiv.org/abs/2410.08405v1
• Date: Thu, 10 Oct 2024 22:38:26 GMT
• Title: AgroGPT: Efficient Agricultural Vision-Language Model with Expert Tuning
• Authors: Muhammad Awais, Ali Husain Salem Abdulla Alharthi, Amandeep Kumar, Hisham Cholakkal, Rao Muhammad Anwer,
• Abstract summary: We propose an approach to construct instruction-tuning data that harnesses vision-only data for the agriculture domain. We utilize diverse agricultural datasets spanning multiple domains, curate class-specific information, and employ large language models (LLMs) to construct an expert-tuning set. We expert-tuned and created AgroGPT, an efficient LMM that can hold complex agriculture-related conversations and provide useful insights.
• Score: 30.034193330398292
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Significant progress has been made in advancing large multimodal conversational models (LMMs), capitalizing on vast repositories of image-text data available online. Despite this progress, these models often encounter substantial domain gaps, hindering their ability to engage in complex conversations across new domains. Recent efforts have aimed to mitigate this issue, albeit relying on domain-specific image-text data to curate instruction-tuning data. However, many domains, such as agriculture, lack such vision-language data. In this work, we propose an approach to construct instruction-tuning data that harnesses vision-only data for the agriculture domain. We utilize diverse agricultural datasets spanning multiple domains, curate class-specific information, and employ large language models (LLMs) to construct an expert-tuning set, resulting in a 70k expert-tuning dataset called AgroInstruct. Subsequently, we expert-tuned and created AgroGPT, an efficient LMM that can hold complex agriculture-related conversations and provide useful insights. We also develop AgroEvals for evaluation and compare {AgroGPT's} performance with large open and closed-source models. {AgroGPT} excels at identifying fine-grained agricultural concepts, can act as an agriculture expert, and provides helpful information for multimodal agriculture questions. The code, datasets, and models are available at https://github.com/awaisrauf/agroGPT.

Related papers

• ShizishanGPT: An Agricultural Large Language Model Integrating Tools and Resources [1.1493479235601496]
  ShizishanGPT is an intelligent question answering system for agriculture based on the Retrieval Augmented Generation framework and agent architecture. ShizishanGPT consists of five key modules: including a generic GPT-4 based module for answering general questions; a search engine module that compensates for the problem that the large language model's own knowledge cannot be updated in a timely manner.
  arXiv  Detail & Related papers  (2024-09-20T14:30:45Z)
• GenQA: Generating Millions of Instructions from a Handful of Prompts [67.54980063851605]
  Most public instruction finetuning datasets are relatively small compared to the closed source datasets used to train industry models. In this work, we study methods for generating large instruction datasets from a single prompt. Our dataset meets or exceeds both WizardLM and Ultrachat on both knowledge-intensive leaderboard tasks as well as conversational evaluations.
  arXiv  Detail & Related papers  (2024-06-14T17:44:08Z)
• Generating Diverse Agricultural Data for Vision-Based Farming Applications [74.79409721178489]
  This model is capable of simulating distinct growth stages of plants, diverse soil conditions, and randomized field arrangements under varying lighting conditions. Our dataset includes 12,000 images with semantic labels, offering a comprehensive resource for computer vision tasks in precision agriculture.
  arXiv  Detail & Related papers  (2024-03-27T08:42:47Z)
• Self-Supervised Backbone Framework for Diverse Agricultural Vision Tasks [0.3683202928838613]
  Computer vision in agriculture is game-changing to transform farming into a data-driven, precise, and sustainable industry. Deep learning has empowered agriculture vision to analyze vast, complex visual data, but heavily rely on the availability of large annotated datasets. We propose a lightweight framework utilizing SimCLR, a contrastive learning approach, to pre-train a ResNet-50 backbone on a large, unannotated dataset of real-world agriculture field images.
  arXiv  Detail & Related papers  (2024-03-22T14:46:51Z)
• Query of CC: Unearthing Large Scale Domain-Specific Knowledge from Public Corpora [104.16648246740543]
  We propose an efficient data collection method based on large language models. The method bootstraps seed information through a large language model and retrieves related data from public corpora. It not only collects knowledge-related data for specific domains but unearths the data with potential reasoning procedures.
  arXiv  Detail & Related papers  (2024-01-26T03:38:23Z)
• PhenoBench -- A Large Dataset and Benchmarks for Semantic Image Interpretation in the Agricultural Domain [29.395926321984565]
  We present an annotated dataset and benchmarks for the semantic interpretation of real agricultural fields. Our dataset recorded with a UAV provides high-quality, pixel-wise annotations of crops and weeds, but also crop leaf instances at the same time. We provide benchmarks for various tasks on a hidden test set comprised of different fields.
  arXiv  Detail & Related papers  (2023-06-07T16:04:08Z)
• AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotators [98.11286353828525]
  GPT-3.5 series models have demonstrated remarkable few-shot and zero-shot ability across various NLP tasks. We propose AnnoLLM, which adopts a two-step approach, explain-then-annotate. We build the first conversation-based information retrieval dataset employing AnnoLLM.
  arXiv  Detail & Related papers  (2023-03-29T17:03:21Z)
• Crop Knowledge Discovery Based on Agricultural Big Data Integration [2.597676155371155]
  Agricultural data can be generated through various sources, such as: Internet of Thing (IoT), sensors, satellites, weather stations, robots, farm equipment, agricultural laboratories, farmers, government agencies and agribusinesses. We propose an agricultural data integration method using a constellation schema which is designed to be flexible enough to incorporate other datasets and big data models.
  arXiv  Detail & Related papers  (2020-03-11T00:13:17Z)
• Data Warehouse and Decision Support on Integrated Crop Big Data [0.0]
  We designed and implemented a continental level agricultural data warehouse (ADW) ADW is characterised by its (1) flexible schema; (2) data integration from real agricultural multi datasets; (3) data science and business intelligent support; (4) high performance; (5) high storage; (6) security; (7) governance and monitoring; (8) consistency, availability and partition tolerant; (9) cloud compatibility.
  arXiv  Detail & Related papers  (2020-03-10T00:10:22Z)
• Agriculture-Vision: A Large Aerial Image Database for Agricultural Pattern Analysis [110.30849704592592]
  We present Agriculture-Vision: a large-scale aerial farmland image dataset for semantic segmentation of agricultural patterns. Each image consists of RGB and Near-infrared (NIR) channels with resolution as high as 10 cm per pixel. We annotate nine types of field anomaly patterns that are most important to farmers.
  arXiv  Detail & Related papers  (2020-01-05T20:19:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.