PestVL-Net: Enabling Multimodal Pest Learning via Fine-grained Vision-Language Interaction

• URL: http://arxiv.org/abs/2604.17278v1
• Date: Sun, 19 Apr 2026 06:17:51 GMT
• Title: PestVL-Net: Enabling Multimodal Pest Learning via Fine-grained Vision-Language Interaction
• Abstract summary: Existing techniques struggle to effectively model the key visual and high-level semantic features of pests in a fine-grained manner.<n>We present a novel vision-language framework, PestVL-Net, with two multi-species pest datasets to facilitate fine-grained pest learning.<n>The deep fusion of these complementary visual and textual representations enables fine-grained multimodal pest learning.
• Score: 11.677135930362446
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Effective pest recognition and management are crucial for sustainable agricultural development. However, collecting pest data in real scenarios is often challenging. Compared to other domains, pests exhibit a wide variety of species with complex and diverse morphological characteristics. Existing techniques struggle to effectively model the key visual and high-level semantic features of pests in a fine-grained manner. These limitations hinder the practical application of such methods in real agricultural scenarios. To address these critical challenges, we present a synergistic approach that integrates PestVL-Net, a novel vision-language framework, with two multi-species pest datasets to facilitate fine-grained pest learning. The visual pathway of PestVL-Net utilizes the Recurrent Weighted Key Value (RWKV) architecture, incorporating a saliency-guided adaptive window partitioning scheme to effectively model the fine-grained visual characteristics of pests. Concurrently, the linguistic component generates precise pest semantic descriptions by leveraging Multimodal Large Language Models (MLLMs) priors, critically informed by agricultural expert knowledge and structured via multimodal Chain-of-Thought (CoT) reasoning. The deep fusion of these complementary visual and textual representations enables fine-grained multimodal pest learning. Extensive experimental evaluations on multiple pest datasets validate the superior performance of PestVL-Net, highlighting its potential for effective real-world pest management.

Related papers

• Pest-Thinker: Learning to Think and Reason like Entomologists via Reinforcement Learning [15.053412132295799]
  We introduce Pest-Thinker, a knowledge-driven reinforcement learning framework that enables MLLMs to reason over fine-grained pest morphology.<n>We show that Pest-Thinker substantially improves both in-domain and out-of-domain morphological understanding, marking a step toward expert-level visual reasoning for intelligent agricultural pest analysis.
  arXiv  Detail & Related papers  (2026-05-07T12:30:02Z)
• Enhancing Multimodal In-Context Learning via Inductive-Deductive Reasoning [65.15766304205657]
  In-context learning (ICL) allows large models to adapt to tasks using a few examples, yet its extension to vision-language models (VLMs) remains fragile.<n>Our analysis reveals that the fundamental limitation lies in an inductive gap, models often produce correct answers from flawed reasoning.<n>We introduce a framework that restructures multimodal ICL as a principled inductive-deductive process.
  arXiv  Detail & Related papers  (2026-05-04T09:18:19Z)
• Multi-View Synergistic Learning with Vision-Language Adaption for Low-Resource Biomedical Image Classification [59.24009931000134]
  Multi-View Synergistic Learning (MVSL) is a unified framework that addresses adaptation paradigms, representation granularity, and disease semantic relationships.<n>MVSL decouples the adaptation of visual and textual encoders to respect their distinct representational characteristics.<n>It further introduces multi-granularity contrastive learning to explicitly model both global image semantics and localized lesion-level evidence.<n>MVSL consistently outperforms state-of-the-art methods in few-shot and zero-shot classification settings.
  arXiv  Detail & Related papers  (2026-04-27T02:41:27Z)
• AgriChat: A Multimodal Large Language Model for Agriculture Image Understanding [13.665861251747144]
  We present AgriChat, a specialized MLLM that presents broad knowledge across thousands of agricultural classes and provides detailed agricultural assessments with explanations.<n>The results validate that preserving visual detail combined with web-verified knowledge constitutes a reliable pathway toward robust and trustworthy agricultural AI.
  arXiv  Detail & Related papers  (2026-03-14T11:49:05Z)
• LVLM-Aided Alignment of Task-Specific Vision Models [49.96265491629163]
  Small task-specific vision models are crucial in high-stakes domains.<n>We introduce a novel and efficient method for aligning small task-specific vision models with human domain knowledge.<n>Our method demonstrates substantial improvement in aligning model behavior with human specifications.
  arXiv  Detail & Related papers  (2025-12-26T11:11:25Z)
• When Alignment Fails: Multimodal Adversarial Attacks on Vision-Language-Action Models [75.16145284285456]
  We introduce VLA-Fool, a comprehensive study of multimodal adversarial robustness in embodied VLA models under both white-box and black-box settings.<n>We develop the first automatically crafted and semantically guided prompting framework.<n> Experiments on the LIBERO benchmark reveal that even minor multimodal perturbations can cause significant behavioral deviations.
  arXiv  Detail & Related papers  (2025-11-20T10:14:32Z)
• MSFNet-CPD: Multi-Scale Cross-Modal Fusion Network for Crop Pest Detection [3.5148549831413036]
  Accurate identification of agricultural pests is essential for crop protection.<n>While deep learning has advanced pest detection, most existing approaches rely solely on low-level visual features.
  arXiv  Detail & Related papers  (2025-05-05T08:10:22Z)
• HiVG: Hierarchical Multimodal Fine-grained Modulation for Visual Grounding [80.85164509232261]
  HiVG consists of a multi-layer adaptive cross-modal bridge and a hierarchical multimodal low-rank adaptation (HiLoRA) paradigm. HiLoRA prevents the accumulation of perceptual errors by adapting the cross-modal features from shallow to deep layers in a hierarchical manner.
  arXiv  Detail & Related papers  (2024-04-20T14:57:31Z)
• InsectMamba: Insect Pest Classification with State Space Model [8.470757741028661]
  InsectMamba is a novel approach that integrates State Space Models (SSMs), Convolutional Neural Networks (CNNs), Multi-Head Self-Attention mechanism (MSA) and Multilayer Perceptrons (MLPs) within Mix-SSM blocks. It was evaluated against strong competitors across five insect pest classification datasets.
  arXiv  Detail & Related papers  (2024-04-04T17:34:21Z)
• A Multimodal Approach for Advanced Pest Detection and Classification [0.9003384937161055]
  This paper presents a novel multi modal deep learning framework for enhanced agricultural pest detection. It combines tiny-BERT's natural language processing with R-CNN and ResNet-18's image processing.
  arXiv  Detail & Related papers  (2023-12-18T05:54:20Z)
• Explainable AI in Grassland Monitoring: Enhancing Model Performance and Domain Adaptability [0.6131022957085438]
  Grasslands are known for their high biodiversity and ability to provide multiple ecosystem services. Challenges in automating the identification of indicator plants are key obstacles to large-scale grassland monitoring. This paper delves into the latter two challenges, with a specific focus on transfer learning and XAI approaches to grassland monitoring.
  arXiv  Detail & Related papers  (2023-12-13T10:17:48Z)
• Delving into Multimodal Prompting for Fine-grained Visual Classification [57.12570556836394]
  Fine-grained visual classification (FGVC) involves categorizing fine subdivisions within a broader category. Recent advancements in pre-trained vision-language models have demonstrated remarkable performance in various high-level vision tasks. We propose a novel multimodal prompting solution, denoted as MP-FGVC, based on the contrastive language-image subcategory (CLIP) model.
  arXiv  Detail & Related papers  (2023-09-16T07:30:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.