BiomedAP: A Vision-Informed Dual-Anchor Framework with Gated Cross-Modal Fusion for Robust Medical Vision-Language Adaptation

• URL: http://arxiv.org/abs/2605.15736v1
• Date: Fri, 15 May 2026 08:45:57 GMT
• Title: BiomedAP: A Vision-Informed Dual-Anchor Framework with Gated Cross-Modal Fusion for Robust Medical Vision-Language Adaptation
• Abstract summary: Biomedical Vision--Language Models (VLMs) have shown remarkable promise in few-shot medical diagnosis.<n>Existing frameworks typically optimize visual and textual prompts as independent streams.<n>BiomedAP is a vision-informed dual-anchor framework with gated cross-modal fusion.
• Score: 13.030685683550317
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Biomedical Vision--Language Models (VLMs) have shown remarkable promise in few-shot medical diagnosis but face a critical bottleneck: \textit{fragility to prompt variations}.Existing adaptation frameworks typically optimize visual and textual prompts as independent streams, relying on ideal ``Golden Prompts''. In clinical reality, where descriptions are often noisy and heterogeneous, this modality isolation leads to unstable cross-modal alignment. To address this, we propose BiomedAP, a vision-informed dual-anchor framework with gated cross-modal fusion.BiomedAP enforces synergistic alignment through two mechanisms: (1) Gated Cross-Modal Fusion, which enables layer-wise interaction between modalities, acting as a dynamic noise regulator to suppress irrelevant textual cues; and (2) a Dual-Anchor Constraint that regularizes learnable prompts toward stable semantic centroids derived from both expert templates (High Anchors) and few-shot visual prototypes (Low Anchors). Extensive experiments across 11 benchmarks demonstrate that BiomedAP consistently surpasses baselines, achieving competitive few-shot accuracy and markedly enhanced robustness under prompt perturbations. Our code is available at: https://github.com/tongdiedie/BiomedAP. Keywords: Vision-Language Models; Prompt Learning; Parameter-Efficient Fine-Tuning; Few-shot Learning

Related papers

• AdaSurvMamba: Dynamic Fusion and Semantic Scanning for Multimodal Survival Analysis [57.942670844512016]
  We introduce AdaSurvMamba as a novel adaptive framework for multimodal survival analysis.<n>The framework features a Dual-Scale Importance-Aware Reconstruction (DSIR) module to dynamically modulate cross-modal interaction strength.<n>Experiments across five TCGA cohorts demonstrate consistent gains over existing methods.
  arXiv  Detail & Related papers  (2026-06-28T11:36:31Z)
• Analyzing and Improving Fine-grained Preference Optimization in Medical LVLMs [53.697403481143404]
  Large Vision-Language Models (LVLMs) have achieved strong performance across medical imaging tasks.<n>But they remain prone to factual inconsistencies, poor visual grounding, and misalignment with clinically meaningful feedback.
  arXiv  Detail & Related papers  (2026-06-10T18:35:36Z)
• Multi-View Synergistic Learning with Vision-Language Adaption for Low-Resource Biomedical Image Classification [59.24009931000134]
  Multi-View Synergistic Learning (MVSL) is a unified framework that addresses adaptation paradigms, representation granularity, and disease semantic relationships.<n>MVSL decouples the adaptation of visual and textual encoders to respect their distinct representational characteristics.<n>It further introduces multi-granularity contrastive learning to explicitly model both global image semantics and localized lesion-level evidence.<n>MVSL consistently outperforms state-of-the-art methods in few-shot and zero-shot classification settings.
  arXiv  Detail & Related papers  (2026-04-27T02:41:27Z)
• DiA-gnostic VLVAE: Disentangled Alignment-Constrained Vision Language Variational AutoEncoder for Robust Radiology Reporting with Missing Modalities [3.5045368873011924]
  We propose the DiA-gnostic VLVAE, which achieves robust radiology reporting through Disentangled Alignment.<n>Our framework is designed to be resilient to missing modalities by disentangling shared and modality-specific features.<n>A compact LLaMA-X decoder then uses these disentangled representations to generate reports efficiently.
  arXiv  Detail & Related papers  (2025-11-08T11:08:27Z)
• TS-P$^2$CL: Plug-and-Play Dual Contrastive Learning for Vision-Guided Medical Time Series Classification [26.293489194966003]
  We introduce a vision-guided paradigm that transforms 1D physiological signals into 2D pseudo-images, establishing a bridge to the visual domain.<n>We employ a dual-contrastive learning strategy, while cross-modal alignment aligns time-series dynamics with visual semantics.<n>Experiments on six MedTS datasets demonstrate that TS-P$2$CL consistently outperforms fourteen methods in both subject-dependent and subject-independent settings.
  arXiv  Detail & Related papers  (2025-09-22T13:57:58Z)
• XBusNet: Text-Guided Breast Ultrasound Segmentation via Multimodal Vision-Language Learning [0.0]
  XBusNet is a novel dual-prompt, dual-branch multimodal model that combines image features with clinically grounded text.<n>We evaluate it on the Breast Lesions USG (BLU) dataset using five-fold cross-validation.<n>Small lesions show the largest gains, with fewer missed regions and fewer spurious activations.
  arXiv  Detail & Related papers  (2025-09-08T20:45:55Z)
• Toward Robust Medical Fairness: Debiased Dual-Modal Alignment via Text-Guided Attribute-Disentangled Prompt Learning for Vision-Language Models [34.42749153279352]
  Vision-language models (VLMs) exhibit strong generalization, and text prompts encode identity attributes.<n>We propose DualFairVL, a multimodal prompt-learning framework that jointly debiases and aligns cross-modal representations.<n>Experiments show that DualFairVL achieves state-of-the-art fairness and accuracy under both in- and out-of-distribution settings.
  arXiv  Detail & Related papers  (2025-08-26T10:01:23Z)
• NEARL-CLIP: Interacted Query Adaptation with Orthogonal Regularization for Medical Vision-Language Understanding [51.63264715941068]
  textbfNEARL-CLIP (iunderlineNteracted quunderlineEry underlineAdaptation with ounderlineRthogonaunderlineL Regularization) is a novel cross-modality interaction VLM-based framework.
  arXiv  Detail & Related papers  (2025-08-06T05:44:01Z)
• DualPrompt-MedCap: A Dual-Prompt Enhanced Approach for Medical Image Captioning [5.456249017636404]
  We present DualPrompt-MedCap, a novel dual-prompt enhancement framework that augments Large Vision-Language Models (LVLMs)<n>A modality-aware prompt derived from a semi-supervised classification model pretrained on medical question-answer pairs, and a question-guided prompt leveraging biomedical language model embeddings.<n>Our method enables the generation of clinically accurate reports that can serve as medical experts' prior knowledge and automatic annotations for downstream vision-language tasks.
  arXiv  Detail & Related papers  (2025-04-13T14:31:55Z)
• PMT: Progressive Mean Teacher via Exploring Temporal Consistency for Semi-Supervised Medical Image Segmentation [51.509573838103854]
  We propose a semi-supervised learning framework, termed Progressive Mean Teachers (PMT), for medical image segmentation. Our PMT generates high-fidelity pseudo labels by learning robust and diverse features in the training process. Experimental results on two datasets with different modalities, i.e., CT and MRI, demonstrate that our method outperforms the state-of-the-art medical image segmentation approaches.
  arXiv  Detail & Related papers  (2024-09-08T15:02:25Z)
• Unify, Align and Refine: Multi-Level Semantic Alignment for Radiology Report Generation [48.723504098917324]
  We propose an Unify, Align and then Refine (UAR) approach to learn multi-level cross-modal alignments. We introduce three novel modules: Latent Space Unifier, Cross-modal Representation Aligner and Text-to-Image Refiner. Experiments and analyses on IU-Xray and MIMIC-CXR benchmark datasets demonstrate the superiority of our UAR against varied state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-03-28T12:42:12Z)
• Learning to Exploit Temporal Structure for Biomedical Vision-Language Processing [53.89917396428747]
  Self-supervised learning in vision-language processing exploits semantic alignment between imaging and text modalities. We explicitly account for prior images and reports when available during both training and fine-tuning. Our approach, named BioViL-T, uses a CNN-Transformer hybrid multi-image encoder trained jointly with a text model.
  arXiv  Detail & Related papers  (2023-01-11T16:35:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.