Analyzing and Improving Fine-grained Preference Optimization in Medical LVLMs

• URL: http://arxiv.org/abs/2606.12590v1
• Date: Wed, 10 Jun 2026 18:35:36 GMT
• Title: Analyzing and Improving Fine-grained Preference Optimization in Medical LVLMs
• Abstract summary: Large Vision-Language Models (LVLMs) have achieved strong performance across medical imaging tasks.<n>But they remain prone to factual inconsistencies, poor visual grounding, and misalignment with clinically meaningful feedback.
• Score: 53.697403481143404
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Vision-Language Models (LVLMs) have achieved strong performance across medical imaging tasks, yet they remain prone to factual inconsistencies, poor visual grounding, and misalignment with clinically meaningful feedback. Existing post-training alignment approaches, including Direct Preference Optimization (DPO) and its variants, face three critical limitations in the medical domain: (1) sequence-level reward signals treat clinically critical tokens identically to generic filler text; (2) reliance on static supervised fine-tuning references as preferred responses introduces an off-policy distribution shift, steering optimization toward stylistic artifacts over clinical correctness; and (3) alignment objectives lack explicit visual grounding constraints, leaving models insensitive to subtle yet diagnostically decisive pathological features. Our method leverages a bidirectional token-wise KL regularizer alongside a visual-contrastive grounding objective that pairs clean and lesion-corrupted images to penalize responses generated without adequate visual evidence. Together, these components form a fine-grained, on-policy alignment framework that constructs preference pairs by minimally editing model-generated outputs, correcting only clinically erroneous spans while preserving the original linguistic style. Extensive experiments across medical imaging tasks and clinical text generation benchmarks validate the effectiveness of our approach.

Related papers

• Towards Clinically Faithful Medical Image Captioning via Enhanced Vision-Language Alignment [10.863931022237757]
  We propose a framework that strengthens clinical alignment by separating and enhancing training-time alignment and inference-time alignment.<n>We build a medical image captioning pipeline that integrates single/dual vision encoders based on BioMedCLIP and SigLIP2, a Q-Former, and a LLaMA-based decoder.<n>Experiments show that complementary visual representations with a multi-encoder design and concept-level auxiliary learning help preserve clinically meaningful information.
  arXiv  Detail & Related papers  (2026-08-20T09:25:30Z)
• Linguistically-Aligned and Visually-Grounded Preference Optimization for Clinically-Augmented Medical Report Generation [15.278073487171502]
  We propose DPO-Clin, a novel post-training framework that focuses preference optimization on clinical findings and cross-modal alignment.<n>ECD module guides the generation of linguistically-aligned report preference pairs.<n>M2DPO variant enforces textual preference inversion triggered by visual context switches.
  arXiv  Detail & Related papers  (2026-08-09T05:29:42Z)
• P$^2$-DPO: Grounding Hallucination in Perceptual Processing via Calibration Direct Preference Optimization [55.74731799669337]
  Hallucination has recently garnered significant research attention in Large Vision-Language Models (LVLMs)<n>Direct Preference Optimization (DPO) aims to learn directly from the corrected preferences provided by humans.<n>Existing preference pairs are often vision-agnostic and their inherently off-policy nature limits their effectiveness in guiding model learning.<n>We propose Perceptual Processing Direct Preference Optimization (P$2$-DPO), a novel training paradigm in which the model generates and learns from its own preference pairs.
  arXiv  Detail & Related papers  (2026-06-02T09:22:53Z)
• Medical Context Distorts Decisions in Clinical Vision Language Models [7.537887356414114]
  Vision-language models (VLMs) are increasingly proposed for clinical decision support, yet their reliability in real-world scenarios remains poorly characterized.<n>This paper identifies three failure modes: modality over-reliance on text over images, (2) spurious reliance on irrelevant clinical history, and (3) prompt sensitivity across semantically equivalent inputs.
  arXiv  Detail & Related papers  (2026-05-17T13:11:38Z)
• ClinCoT: Clinical-Aware Visual Chain-of-Thought for Medical Vision Language Models [24.19721015692576]
  We propose ClinCoT to transform preference optimization from response-level correction to visual-driven reasoning.<n>We show that ClinCoT consistently improves factual grounding and achieves superior performance compared with existing preference-based alignment methods.
  arXiv  Detail & Related papers  (2026-03-01T14:15:54Z)
• Visual Alignment of Medical Vision-Language Models for Grounded Radiology Report Generation [25.148217482604746]
  We propose VALOR:Visual Alignment of Medical Vision-Language Models for Radiology Report Generation.<n>Our method introduces a reinforcement learning-based post-alignment framework utilizing Group-Relative Proximal Optimization (GRPO)<n>Experiments on multiple benchmarks demonstrate that VALOR substantially improves factual accuracy and visual grounding, achieving significant performance gains over state-of-the-art report generation methods.
  arXiv  Detail & Related papers  (2025-12-18T05:48:21Z)
• Enhancing CLIP Robustness via Cross-Modality Alignment [54.01929554563447]
  We propose Cross-modality Alignment, an optimal transport-based framework for vision-language models.<n> COLA restores global image-text alignment and local structural consistency in the feature space.<n> COLA is training-free and compatible with existing fine-tuned models.
  arXiv  Detail & Related papers  (2025-10-28T03:47:44Z)
• MedAlign: A Synergistic Framework of Multimodal Preference Optimization and Federated Meta-Cognitive Reasoning [52.064286116035134]
  We develop MedAlign, a framework to ensure visually accurate LVLM responses for Medical Visual Question Answering (Med-VQA)<n>We first propose a multimodal Direct Preference Optimization (mDPO) objective to align preference learning with visual context.<n>We then design a Retrieval-Aware Mixture-of-Experts (RA-MoE) architecture that utilizes image and text similarity to route queries to a specialized and context-augmented LVLM.
  arXiv  Detail & Related papers  (2025-10-24T02:11:05Z)
• Self-Supervised Anatomical Consistency Learning for Vision-Grounded Medical Report Generation [61.350584471060756]
  Vision-grounded medical report generation aims to produce clinically accurate descriptions of medical images.<n>We propose Self-Supervised Anatomical Consistency Learning (SS-ACL) to align generated reports with corresponding anatomical regions.<n>SS-ACL constructs a hierarchical anatomical graph inspired by the invariant top-down inclusion structure of human anatomy.
  arXiv  Detail & Related papers  (2025-09-30T08:59:06Z)
• CLARIFID: Improving Radiology Report Generation by Reinforcing Clinically Accurate Impressions and Enforcing Detailed Findings [1.515687944002438]
  We propose CLARIFID, a novel framework that directly optimize diagnostic correctness by mirroring the two-step workflow of experts.<n> CLARIFID learns the logical flow from Findings to Impression through section-aware pretraining.<n>We show that our method achieves superior clinical efficacy and outperforms existing baselines on both standard NLG metrics and clinically aware scores.
  arXiv  Detail & Related papers  (2025-07-23T05:57:59Z)
• Rethinking Semi-Supervised Medical Image Segmentation: A Variance-Reduction Perspective [51.70661197256033]
  We propose ARCO, a semi-supervised contrastive learning framework with stratified group theory for medical image segmentation. We first propose building ARCO through the concept of variance-reduced estimation and show that certain variance-reduction techniques are particularly beneficial in pixel/voxel-level segmentation tasks. We experimentally validate our approaches on eight benchmarks, i.e., five 2D/3D medical and three semantic segmentation datasets, with different label settings.
  arXiv  Detail & Related papers  (2023-02-03T13:50:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.