CropVLM: A Domain-Adapted Vision-Language Model for Open-Set Crop Analysis

• URL: http://arxiv.org/abs/2605.03259v1
• Date: Tue, 05 May 2026 01:17:27 GMT
• Title: CropVLM: A Domain-Adapted Vision-Language Model for Open-Set Crop Analysis
• Abstract summary: We present CropVLM, a vision-language model adapted for the agricultural domain via Domain-Specific Semantic Alignment (DSSA)<n>CropVLM is trained on 52,987 manually selected image-caption pairs covering 37 species in natural field conditions.<n>In comprehensive evaluations, CropVLM achieves 72.51% zero-shot classification accuracy, outperforming seven CLIP-style baselines.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: High-throughput plant phenotyping, the quantitative measurement of observable plant traits, is critical for modern breeding but remains constrained by a "phenotyping bottleneck," where manual data collection is labor-intensive and prone to observer bias. Conventional closed-set computer vision systems fail to address this challenge, as they require extensive species-specific annotation and lack the flexibility to handle diverse breeding populations. To bridge this gap, we present CropVLM, a Vision-Language Model (VLM) adapted for the agricultural domain via Domain-Specific Semantic Alignment (DSSA). Trained on 52,987 manually selected image-caption pairs covering 37 species in natural field conditions, CropVLM effectively maps agronomic terminology to fine-grained visual features. We further introduce the Hybrid Open-Set Localization Network (HOS-Net), an architecture that integrates CropVLM to enable the detection of novel crops solely from natural language descriptions without retraining. By eliminating the reliance on species-specific training data, CropVLM provides a scalable solution for high-throughput phenotyping, accelerating genetic gain and facilitating large-scale biodiversity research essential for sustainable agriculture. The trained model weights and complete pipeline implementation are publicly available at: [https://github.com/boudiafA/CropVLM](https://github.com/boudiafA/CropVLM). In comprehensive evaluations, CropVLM achieves 72.51% zero-shot classification accuracy, outperforming seven CLIP-style baselines. Our detection pipeline demonstrates superior zero-shot generalization to novel species, achieving 49.17 AP50 on our CVTCropDet benchmark and 50.73 AP50 on tropical fruit species, compared to 34.89 and 48.58 for the next-best method, respectively.

Related papers

• WeedExpert-R1: Incentivizing Botanical Reasoning in MLLMs with Reinforcement Learning for Precision Weed Grounding [5.155445137665098]
  This study introduces WeedExpert-R1, a multimodal model that learns visually grounded botanical reasoning through verifiable rewards.<n>WeedExpert-R1-4B achieved 75.82 percent exact-set precision at an IoU threshold of 0.5, 89.30 percent precision, and 87.81 percent recall.
  arXiv  Detail & Related papers  (2026-07-17T20:33:10Z)
• DVMap: Fine-Grained Pluralistic Value Alignment via High-Consensus Demographic-Value Mapping [49.402442592047855]
  Current Large Language Models (LLMs) typically rely on coarse-grained national labels for pluralistic value alignment.<n>We argue that resolving this limitation requires shifting from national labels to multi-dimensional demographic constraints.<n>We propose DVMap, a framework for fine-grained pluralistic value alignment.
  arXiv  Detail & Related papers  (2026-05-14T06:06:43Z)
• ReLeaf: Benchmarking Leaf Segmentation across Domains and Species [2.446672595462589]
  Leaf-level segmentation is used to assess crop health, growth dynamics, yield potential and localized stress symptoms.<n>Progress is limited by a lack of dedicated datasets, especially regarding species coverage.<n>We introduce a new benchmark dataset with leaf-level masks for 23 plant species, created via semi-automatic annotation of selected CropAndWeed images.<n>A model trained on all four existing datasets achieves a mean mAP50-95 of 83.9% across their corresponding test sets and 40.2% on our new benchmark.
  arXiv  Detail & Related papers  (2026-05-05T14:11:52Z)
• TagaVLM: Topology-Aware Global Action Reasoning for Vision-Language Navigation [70.23578202012048]
  Vision-Language Navigation (VLN) presents a unique challenge for Large Vision-Language Models (VLMs) due to their inherent architectural mismatch.<n>We propose TagaVLM (Topology-Aware Global Action reasoning), an end-to-end framework that explicitly injects topological structures into the VLM backbone.<n>To enhance topological node information, an Interleaved Navigation Prompt strengthens node-level visual-text alignment.<n>With the embedded topological graph, the model is capable of global action reasoning, allowing for robust path correction.
  arXiv  Detail & Related papers  (2026-03-03T13:28:07Z)
• Vision-Language Semantic Grounding for Multi-Domain Crop-Weed Segmentation [6.56750055693609]
  Fine-grained crop-weed segmentation is essential for enabling targeted herbicide application in precision agriculture.<n>Existing deep learning models struggle to generalize due to reliance on dataset-specific visual features.<n>We propose Vision-Language Weed (VL-WS), a novel framework that addresses this limitation by grounding pixel-level segmentation in semantically aligned, domain-invariant representations.
  arXiv  Detail & Related papers  (2026-02-27T04:53:18Z)
• Are vision-language models ready to zero-shot replace supervised classification models in agriculture? [0.8594140167290097]
  Vision models (VLMs) are proposed as general-purpose solutions for visual recognition tasks.<n>We benchmark a diverse set open and closed-source VLMs on 27 agricultural classification datasets from the AgML collection.
  arXiv  Detail & Related papers  (2025-12-17T21:22:44Z)
• Vision Foundation Models in Agriculture: Toward Domain-Specific Adaptation for Weed Herbicide Trials Assessment [1.8430060563461854]
  Herbicide field trials require accurate identification of plant species and assessment of herbicide-induced damage.<n>In this work, we adapt a general-purpose vision foundation model to herbicide trial characterization.<n> Trained using a self-supervised learning approach on a large, curated agricultural dataset, the model learns rich and transferable representations optimized for herbicide trials images.
  arXiv  Detail & Related papers  (2025-11-06T11:30:32Z)
• Transfer Learning and Mixup for Fine-Grained Few-Shot Fungi Classification [0.0]
  This paper presents our approach for the FungiCLEF 2025 competition.<n>It focuses on few-shot fine-grained visual categorization using the FungiTastic Few-Shot dataset.
  arXiv  Detail & Related papers  (2025-07-11T01:21:21Z)
• BioCLIP 2: Emergent Properties from Scaling Hierarchical Contrastive Learning [60.80381372245902]
  We find emergent behaviors in biological vision models via large-scale contrastive vision-language training.<n>We train BioCLIP 2 on TreeOfLife-200M to distinguish different species.<n>We identify emergent properties in the learned embedding space of BioCLIP 2.
  arXiv  Detail & Related papers  (2025-05-29T17:48:20Z)
• Spatial Implicit Neural Representations for Global-Scale Species Mapping [72.92028508757281]
  Given a set of locations where a species has been observed, the goal is to build a model to predict whether the species is present or absent at any location. Traditional methods struggle to take advantage of emerging large-scale crowdsourced datasets. We use Spatial Implicit Neural Representations (SINRs) to jointly estimate the geographical range of 47k species simultaneously.
  arXiv  Detail & Related papers  (2023-06-05T03:36:01Z)
• Improving Visual Grounding by Encouraging Consistent Gradient-based Explanations [58.442103936918805]
  We show that Attention Mask Consistency produces superior visual grounding results than previous methods. AMC is effective, easy to implement, and is general as it can be adopted by any vision-language model.
  arXiv  Detail & Related papers  (2022-06-30T17:55:12Z)
• Two-View Fine-grained Classification of Plant Species [66.75915278733197]
  We propose a novel method based on a two-view leaf image representation and a hierarchical classification strategy for fine-grained recognition of plant species. A deep metric based on Siamese convolutional neural networks is used to reduce the dependence on a large number of training samples and make the method scalable to new plant species.
  arXiv  Detail & Related papers  (2020-05-18T21:57:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.