Context Blindness in DPO: Mitigating Object Hallucination in MLLMs via Context-Calibrated Preference Optimization

• URL: http://arxiv.org/abs/2608.12158v1
• Date: Wed, 12 Aug 2026 15:20:29 GMT
• Title: Context Blindness in DPO: Mitigating Object Hallucination in MLLMs via Context-Calibrated Preference Optimization
• Abstract summary: We propose a metric that measures how much a model's preference strengthens when relevant context is provided.<n>We find that higher CPG consistently corresponds to lower hallucination, yet standard DPO and its variants exhibit only limited CPG.<n>We propose Context-Calibrated DPO (C$2$-DPO), which directly maximizes CPG while preserving the original preference ordering.
• Score: 34.25772615253414
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal large language models (MLLMs) have made rapid progress, yet they still exhibit object hallucination, generating plausible but incorrect descriptions that are inconsistent with the visual input. Direct Preference Optimization (DPO) mitigates this by training models to prefer non-hallucinated responses over hallucinated ones, and recent efforts further enrich the preference data with relevant context. However, it remains unclear whether DPO actually leverages such context. To investigate this, we propose Contextual Preference Gain (CPG), a simple metric that measures how much a model's preference strengthens when relevant context is provided. We find that higher CPG consistently corresponds to lower hallucination, yet standard DPO and its variants exhibit only limited CPG, indicating that they underutilize contextual information and thus remain prone to hallucination. To address this, we propose Context-Calibrated DPO (C$^2$-DPO), which directly maximizes CPG while preserving the original preference ordering. Across multiple benchmarks, C$^2$-DPO substantially reduces hallucination without compromising general reasoning, relatively reducing the Object HalBench hallucination rate of Qwen2-VL-Instruct-2B by 36%. Code is available at https://github.com/mlvlab/C2-DPO

Related papers

• Mult-DPO: Multinomial Direct Preference Optimization for Recommender Systems [72.27018654934386]
  Direct preference optimization (DPO) is a simple and effective alignment strategy for large language models (LLMs) based on pairwise preferences.<n>In recommender systems, however, user feedback is rarely pairwise.<n>We propose Mult-DPO, a tractable multinomial surrogate likelihood over set-wise preference events for the user-preference alignment of LLM-based recommender systems.
  arXiv  Detail & Related papers  (2026-06-08T18:53:45Z)
• P$^2$-DPO: Grounding Hallucination in Perceptual Processing via Calibration Direct Preference Optimization [55.74731799669337]
  Hallucination has recently garnered significant research attention in Large Vision-Language Models (LVLMs)<n>Direct Preference Optimization (DPO) aims to learn directly from the corrected preferences provided by humans.<n>Existing preference pairs are often vision-agnostic and their inherently off-policy nature limits their effectiveness in guiding model learning.<n>We propose Perceptual Processing Direct Preference Optimization (P$2$-DPO), a novel training paradigm in which the model generates and learns from its own preference pairs.
  arXiv  Detail & Related papers  (2026-06-02T09:22:53Z)
• When Prompts Override Vision: Prompt-Induced Hallucinations in LVLMs [54.411658510110215]
  We propose HalluScope to better understand the extent to which different factors induce hallucinations.<n>HalluVL-DPO is a framework for fine-tuning off-the-shelf LVLMs towards more visually grounded responses.
  arXiv  Detail & Related papers  (2026-04-23T17:54:36Z)
• Mitigating Hallucinations in Multimodal LLMs via Object-aware Preference Optimization [55.543583937522804]
  Multimodal Large Language Models (MLLMs) emerge as a unified interface to address a multitude of tasks.<n>Despite showcasing state-of-the-art results in many benchmarks, a long-standing issue is the tendency of MLLMs to hallucinate.<n>In this paper, we address the problem of hallucinations as an alignment problem, seeking to steer the MLLM so that it prefers generating content without hallucinations.
  arXiv  Detail & Related papers  (2025-08-27T18:02:04Z)
• TARS: MinMax Token-Adaptive Preference Strategy for MLLM Hallucination Reduction [17.776737597391843]
  Multimodal large language models (MLLMs) enable vision- reasoning, yet often generate plausible outputs that are factually incorrect or visually ungrounded.<n>Direct preference optimization (DPO) is a common strategy for correcting hallucinations by aligning model outputs with human preferences.<n>We propose TARS, a token-adaptive preference strategy that reformulates DPO as a min-max optimization problem.
  arXiv  Detail & Related papers  (2025-07-29T08:39:19Z)
• CHiP: Cross-modal Hierarchical Direct Preference Optimization for Multimodal LLMs [107.21334626890713]
  Multimodal Large Language Models (MLLMs) still struggle with hallucinations despite their impressive capabilities.<n>We propose a Cross-modal Hierarchical Direct Preference Optimization (CHiP) to address these limitations.<n>We evaluate CHiP through both quantitative and qualitative analyses, with results across multiple benchmarks demonstrating its effectiveness in reducing hallucinations.
  arXiv  Detail & Related papers  (2025-01-28T02:05:38Z)
• Mitigating Hallucinations in Large Vision-Language Models via DPO: On-Policy Data Hold the Key [24.229983103296988]
  Hallucination remains a major challenge for Large Vision-Language Models (LVLMs)<n>We propose On-Policy Alignment (OPA)-DPO framework, which uniquely leverages expert feedback to correct hallucinated responses.<n>OPA-DPO achieves an additional reduction in the hallucination rate of LLaVA-1.5-7B: 13.26% on the AMBER benchmark and 5.39% on the Object-Hal benchmark.
  arXiv  Detail & Related papers  (2025-01-16T17:48:03Z)
• Aligning Modalities in Vision Large Language Models via Preference Fine-tuning [67.62925151837675]
  In this work, we frame the hallucination problem as an alignment issue, tackle it with preference tuning. Specifically, we propose POVID to generate feedback data with AI models. We use ground-truth instructions as the preferred response and a two-stage approach to generate dispreferred data. In experiments across broad benchmarks, we show that we can not only reduce hallucinations, but improve model performance across standard benchmarks, outperforming prior approaches.
  arXiv  Detail & Related papers  (2024-02-18T00:56:16Z)
• Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimization [45.53216822981202]
  This paper introduces a novel solution, Hallucination-Aware Direct Preference Optimization (HA-DPO), which reframes the hallucination problem as a preference selection task. When applied to three mainstream multimodal models, HA-DPO significantly reduced hallucination issues and amplified the models' generalization capabilities.
  arXiv  Detail & Related papers  (2023-11-28T14:54:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.