AMANDA: Agentic Medical Knowledge Augmentation for Data-Efficient Medical Visual Question Answering

• URL: http://arxiv.org/abs/2510.02328v1
• Date: Fri, 26 Sep 2025 01:22:25 GMT
• Title: AMANDA: Agentic Medical Knowledge Augmentation for Data-Efficient Medical Visual Question Answering
• Authors: Ziqing Wang, Chengsheng Mao, Xiaole Wen, Yuan Luo, Kaize Ding,
• Abstract summary: We propose AMANDA, a training-free agentic framework that performs medical knowledge augmentation via LLM agents.<n>Experiments across eight Med-VQA benchmarks demonstrate substantial improvements in both zero-shot and few-shot Med-VQA settings.
• Score: 34.90463380394591
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Medical Multimodal Large Language Models (Med-MLLMs) have shown great promise in medical visual question answering (Med-VQA). However, when deployed in low-resource settings where abundant labeled data are unavailable, existing Med-MLLMs commonly fail due to their medical reasoning capability bottlenecks: (i) the intrinsic reasoning bottleneck that ignores the details from the medical image; (ii) the extrinsic reasoning bottleneck that fails to incorporate specialized medical knowledge. To address those limitations, we propose AMANDA, a training-free agentic framework that performs medical knowledge augmentation via LLM agents. Specifically, our intrinsic medical knowledge augmentation focuses on coarse-to-fine question decomposition for comprehensive diagnosis, while extrinsic medical knowledge augmentation grounds the reasoning process via biomedical knowledge graph retrieval. Extensive experiments across eight Med-VQA benchmarks demonstrate substantial improvements in both zero-shot and few-shot Med-VQA settings. The code is available at https://github.com/REAL-Lab-NU/AMANDA.

Related papers

• MedREK: Retrieval-Based Editing for Medical LLMs with Key-Aware Prompts [70.64143198545031]
  We propose MedREK, a retrieval-based editing framework that integrates a shared query-key module for precise matching with an attention-based prompt encoder for informative guidance.<n>Our results on various medical benchmarks demonstrate that our MedREK achieves superior performance across different core metrics.
  arXiv  Detail & Related papers  (2025-10-15T12:50:33Z)
• Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning [57.873833577058]
  We build a multimodal dataset enriched with extensive medical knowledge.<n>We then introduce our medical-specialized MLLM: Lingshu.<n>Lingshu undergoes multi-stage training to embed medical expertise and enhance its task-solving capabilities.
  arXiv  Detail & Related papers  (2025-06-08T08:47:30Z)
• Fact or Guesswork? Evaluating Large Language Models' Medical Knowledge with Structured One-Hop Judgments [108.55277188617035]
  Large language models (LLMs) have been widely adopted in various downstream task domains, but their abilities to directly recall and apply factual medical knowledge remains under-explored.<n>We introduce the Medical Knowledge Judgment dataset (MKJ), a dataset derived from the Unified Medical Language System (UMLS), a comprehensive repository of standardized vocabularies and knowledge graphs.<n>Through a binary classification framework, MKJ evaluates LLMs' grasp of fundamental medical facts by having them assess the validity of concise, one-hop statements.
  arXiv  Detail & Related papers  (2025-02-20T05:27:51Z)
• ClinKD: Cross-Modal Clinical Knowledge Distiller For Multi-Task Medical Images [4.353855760968461]
  Cross-Modal Clinical Knowledge Distiller (ClinKD) designed to enhance image-text alignment and establish more effective medical knowledge transformation mechanisms.<n>ClinKD achieves state-of-the-art performance on several datasets which are challenging for Med-VQA task.
  arXiv  Detail & Related papers  (2025-02-09T15:08:10Z)
• Improving Retrieval-Augmented Generation in Medicine with Iterative Follow-up Questions [42.73799041840482]
  i-MedRAG is a system that iteratively asks follow-up queries based on previous information-seeking attempts. Our zero-shot i-MedRAG outperforms all existing prompt engineering and fine-tuning methods on GPT-3.5. i-MedRAG can flexibly ask follow-up queries to form reasoning chains, providing an in-depth analysis of medical questions.
  arXiv  Detail & Related papers  (2024-08-01T17:18:17Z)
• MultifacetEval: Multifaceted Evaluation to Probe LLMs in Mastering Medical Knowledge [4.8004472307210255]
  Large language models (LLMs) have excelled across domains, delivering notable performance on medical evaluation benchmarks. However, there still exists a significant gap between the reported performance and the practical effectiveness in real-world medical scenarios. We develop a novel evaluation framework MultifacetEval to examine the degree and coverage of LLMs in encoding and mastering medical knowledge.
  arXiv  Detail & Related papers  (2024-06-05T04:15:07Z)
• MedKP: Medical Dialogue with Knowledge Enhancement and Clinical Pathway Encoding [48.348511646407026]
  We introduce the Medical dialogue with Knowledge enhancement and clinical Pathway encoding framework. The framework integrates an external knowledge enhancement module through a medical knowledge graph and an internal clinical pathway encoding via medical entities and physician actions.
  arXiv  Detail & Related papers  (2024-03-11T10:57:45Z)
• OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLM [48.16696073640864]
  We introduce OmniMedVQA, a novel comprehensive medical Visual Question Answering (VQA) benchmark. All images in this benchmark are sourced from authentic medical scenarios. We have found that existing LVLMs struggle to address these medical VQA problems effectively.
  arXiv  Detail & Related papers  (2024-02-14T13:51:56Z)
• Med-Flamingo: a Multimodal Medical Few-shot Learner [58.85676013818811]
  We propose Med-Flamingo, a multimodal few-shot learner adapted to the medical domain. Based on OpenFlamingo-9B, we continue pre-training on paired and interleaved medical image-text data from publications and textbooks. We conduct the first human evaluation for generative medical VQA where physicians review the problems and blinded generations in an interactive app.
  arXiv  Detail & Related papers  (2023-07-27T20:36:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.