Fusing Perceptual Vision Experts with Multimodal Large Language Models for Explainable Plant Disease Diagnosis: From Benchmark Imagery to Real-World Robotic Field Validation

• URL: http://arxiv.org/abs/2608.24934v1
• Date: Sun, 23 Aug 2026 17:20:20 GMT
• Title: Fusing Perceptual Vision Experts with Multimodal Large Language Models for Explainable Plant Disease Diagnosis: From Benchmark Imagery to Real-World Robotic Field Validation
• Abstract summary: We present the Hybrid Hierarchical Multi-Agent Framework (H$2$MAF), combining decision-level fusion of EfficientNet-B3 and ConvNeXt-Tiny with semantic arbitration.<n>On PlantDoc, Gemma improves accuracy from 63.9% to 68.5%, achieving +7.6 points on the 41.7% CNN-conflict subset.<n>These results establish MLLM arbitration as a promising, yet calibration-dependent, approach for explainable agricultural AI and robotic field decision support.
• Score: 3.7105561711383266
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accurate field plant disease diagnosis requires reliable fusion of uncertain and conflicting perceptual evidence. We present the Hybrid Hierarchical Multi-Agent Framework (H$^{2}$MAF), combining decision-level fusion of EfficientNet-B3 and ConvNeXt-Tiny with semantic arbitration by open-weight multimodal large language models (MLLMs), Gemma 4 E4B and Qwen3.5 4B, using structured JSON evidence to generate explainable diagnoses, risk levels, treatment urgency, and financial exposure. (H$^{2}$MAF) is evaluated on 14,364 images (1,370 test images) across PlantDoc (2,922 images, 27 classes) and two non-public, continuously captured Cornell robot-acquired field datasets: Stage 2 (20 GB; 4,215 images) and Stage 4 (40 GB; 7,227 images), covering Early Blight, Late Blight, and Septoria Leaf Spot under uncontrolled field conditions. On PlantDoc, Gemma improves accuracy from 63.9% to 68.5%, achieving +7.6 points on the 41.7% CNN-conflict subset. Cornell accuracies reach 99.3% and 98.9%, with only 1.7-4.1% disagreement, demonstrating conflict-dependent MLLM utility. The critical-risk error of gemma is 0.14-0.5 points, whereas Qwen overflags by 3.5-14.4 points. These results establish MLLM arbitration as a promising, yet calibration-dependent, approach for explainable agricultural AI and robotic field decision support. Github Link: https://github.com/Applied-AI-Research-Lab/Explainable-AI-Plant-Disease-Detection

Related papers

• Not All Relations Are Equal: Relation-Balanced and Calibrated Graph Learning for Provenance-Based Intrusion Detection [66.44527094471619]
  We present RECAL, an unsupervised framework using relation-balanced masked graph learning to better capture rare interaction patterns.<n>On three DARPA E3 datasets, RECAL achieves F1 scores of 99.99%, 99.93%, and 99.99%, outperforming the best baseline on each dataset by 0.88, 0.82, and 0.42 percentage points, respectively.
  arXiv  Detail & Related papers  (2026-09-15T00:31:01Z)
• An Agentic AI Framework Overcomes Fundamental Limitations of Large Language Models for Glaucoma Detection from Fundus Photography [1.3023698730561286]
  Large language models (LLMs) show promise in medical image interpretation but suffer from hallucination, limited accuracy, and run-to-run inconsistency.<n>We developed and validated an agentic AI framework integrating LLMs with specialized deep learning tools for glaucoma detection from fundus photography.
  arXiv  Detail & Related papers  (2026-08-07T17:33:12Z)
• KAISEN: Reproducible Subgroup Fairness Auditing for Clinical Risk Models [0.0]
  KAISEN is a five-phase audit pipeline covering subgroup stratification, disparity measurement, mechanism diagnostics, post-hoc mitigation, and drift monitoring.<n>All results are synthetic with known ground truth and do not establish clinical validity.
  arXiv  Detail & Related papers  (2026-07-30T17:57:18Z)
• Theia: Large-Scale Multimodal Captioning and Automated Validation of the Incidents1M Dataset for Data-Free Distillation [43.1212452324751]
  Existing datasets in this domain either entirely lack descriptive text, or suffer from severe text-image semantic misalignment.<n>We present a novel methodology to construct and automatically validate a large-scale multimodal dataset for disaster response.
  arXiv  Detail & Related papers  (2026-07-30T14:23:01Z)
• A Controlled Audit of Pretraining Contamination in Public Medical Vision-Language Benchmarks [46.43161388634735]
  Medical vision-language models (VLMs) are evaluated on public benchmarks whose images and question-answer pairs have been freely downloadable for years.<n>We audit open VLMs on SLAKE-En, PathVQA, VQA-RAD, and an auxiliary public OmniMedVQA mirror using four detector families.
  arXiv  Detail & Related papers  (2026-06-08T18:40:10Z)
• Medical thinking with multiple images [44.04557445622649]
  We introduce MedThinkVQA, an expert-annotated benchmark for thinking with multiple images.<n>The dataset contains 8,067 cases, including 720 test cases, with an average of 6.62 images per case.<n>On the test set, the best closed-source models, Claude-4.6-Opus, Gemini-3-Pro, and GPT-5.2-xhigh, reach only 57.2%, 55.3%, and 54.9% accuracy.
  arXiv  Detail & Related papers  (2026-04-14T18:51:07Z)
• Predictive Entropy Links Calibration and Paraphrase Sensitivity in Medical Vision-Language Models [2.064612766965483]
  We show five uncertainty quantification methods on MedGemma 4BIT across in distribution MIMIC CXR and outof distribution PadChest chest X ray datasets, with cross architecture validation on LLaVA RAD7B.<n>For well calibrated single model methods, predictive entropy from one forward pass predicts which samples will flip under rephrasing AUROC 0.711 on MedGemma, 0.878 on LLaVARAD p 10 4, enabling a single entropy threshold to flag both unreliable and sensitive predictions.
  arXiv  Detail & Related papers  (2026-04-10T04:18:47Z)
• Fragile Reasoning: A Mechanistic Analysis of LLM Sensitivity to Meaning-Preserving Perturbations [2.7946918847372277]
  Large language models demonstrate strong performance on mathematical reasoning benchmarks, yet remain surprisingly fragile to meaning-preserving surface perturbations.<n>We systematically evaluate three open-weight LLMs, Mistral-7B, Llama-3-8B, and Qwen2.5-7B, on 677 GSM8K problems paired with semantically equivalent variants.<n>All three models exhibit substantial answer-flip rates (28.8%-45.1%), with number paraphrasing consistently more disruptive than name swaps.
  arXiv  Detail & Related papers  (2026-04-02T05:30:20Z)
• AgriPath: A Systematic Exploration of Architectural Trade-offs for Crop Disease Classification [45.52399819498234]
  This work presents a systematic empirical comparison of three model paradigms for fine-grained crop disease classification.<n>We introduce AgriPath-LF16, a benchmark containing 111k images spanning 16 crops and 41 diseases.<n>CNNs achieve the highest accuracy on lab imagery but degrade under domain shift.<n> Contrastive VLMs provide a robust and parameter-efficient alternative with competitive cross-domain performance.
  arXiv  Detail & Related papers  (2026-03-08T17:28:01Z)
• Exploring Response Uncertainty in MLLMs: An Empirical Evaluation under Misleading Scenarios [49.53589774730807]
  Multimodal large language models (MLLMs) have recently achieved state-of-the-art performance on tasks ranging from visual question answering to video understanding.<n>We reveal a response uncertainty phenomenon: twelve state-of-the-art open-source MLLMs overturn a previously correct answer in 65% of cases after receiving a single deceptive cue.
  arXiv  Detail & Related papers  (2024-11-05T01:11:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.