When Relations Break: Analyzing Relation Hallucination in Vision-Language Model Under Rotation and Noise

• URL: http://arxiv.org/abs/2605.05045v2
• Date: Sun, 10 May 2026 12:13:49 GMT
• Title: When Relations Break: Analyzing Relation Hallucination in Vision-Language Model Under Rotation and Noise
• Abstract summary: Vision-language models (VLMs) achieve strong multimodal performance but remain prone to relation hallucination, which requires accurate reasoning over inter-object interactions.<n>We study the impact of visual perturbations, specifically rotation and noise, and show that even mild distortions significantly degrade relational reasoning across models and datasets.
• Score: 11.519569849803757
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-language models (VLMs) achieve strong multimodal performance but remain prone to relation hallucination, which requires accurate reasoning over inter-object interactions. We study the impact of visual perturbations, specifically rotation and noise, and show that even mild distortions significantly degrade relational reasoning across models and datasets. We further evaluate prompt-based augmentation and preprocessing strategies (orientation correction and denoising), finding that while they offer partial improvements, they do not fully resolve hallucinations. Our results reveal a gap between perceptual robustness and relational understanding, highlighting the need for more robust, geometry-aware VLMs.

Related papers

• Cognitive Pivot Points and Visual Anchoring: Unveiling and Rectifying Hallucinations in Multimodal Reasoning Models [65.57985131861399]
  hallucinations are strongly correlated with cognitive bifurcation points that often exhibit high entropy states.<n>We propose V-STAR, a lightweight, holistic training paradigm designed to internalize visually aware reasoning capabilities.
  arXiv  Detail & Related papers  (2026-04-11T13:59:05Z)
• ACT Now: Preempting LVLM Hallucinations via Adaptive Context Integration [71.21097024566285]
  Large-Language Models (LVLMs) frequently suffer from severe hallucination issues.<n>Existing mitigation strategies rely on linguistic, single-step states to enhance visual focus or suppress strong priors.<n>We propose Adaptive Context in VisionTegration (ACT), a training-free inference intervention method that mitigates hallucination through the adaptive integration of contextual information.
  arXiv  Detail & Related papers  (2026-04-01T14:49:50Z)
• Looking Back and Forth: Cross-Image Attention Calibration and Attentive Preference Learning for Multi-Image Hallucination Mitigation [48.06125338526647]
  Large vision-language models (LVLMs) are prone to hallucinations in multi-image tasks.<n>We propose a structured hallucination mitigation framework involving Cross-Image Attention calibration and Preference Learning.
  arXiv  Detail & Related papers  (2026-03-07T05:57:27Z)
• Seeing Through the Chain: Mitigate Hallucination in Multimodal Reasoning Models via CoT Compression and Contrastive Preference Optimization [78.94590726578014]
  multimodal reasoning models (MLRMs) remain prone to hallucinations, and effective solutions are still underexplored.<n>We propose C3PO, a training-based mitigation framework comprising textbfCompression and textbfPreference textbfOptimization.
  arXiv  Detail & Related papers  (2026-02-03T11:00:55Z)
• IKOD: Mitigating Visual Attention Degradation in Large Vision-Language Models [20.036659182106806]
  We show that Large Vision-Language Models (LVLMs) exhibit a long-term bias where hallucinations increase as the sequence length grows.<n>We propose Image attention-guided Key-value merging cOllaborative Decoding (IKOD), a collaborative decoding strategy generating more image-focused sequences.
  arXiv  Detail & Related papers  (2025-08-05T14:05:15Z)
• Mitigating Hallucinations in Large Vision-Language Models by Adaptively Constraining Information Flow [32.039946174953236]
  Large vision-language models show tremendous potential in understanding visual information through human languages.<n>They are prone to suffer from object hallucination, i.e., the generated image descriptions contain objects that do not exist in the image.<n>We propose Variational Information Bottleneck (VIB) to alleviate overconfidence by introducing hallucination noise.
  arXiv  Detail & Related papers  (2025-02-28T05:56:23Z)
• Mitigating Hallucination for Large Vision Language Model by Inter-Modality Correlation Calibration Decoding [66.06337890279839]
  Large vision-language models (LVLMs) have shown remarkable capabilities in visual-language understanding for downstream multi-modal tasks.<n>LVLMs still suffer from generating hallucinations in complex generation tasks, leading to inconsistencies between visual inputs and generated content.<n>We propose an Inter-Modality Correlation Decoding (IMCCD) method to mitigate hallucinations in LVLMs in a training-free manner.
  arXiv  Detail & Related papers  (2025-01-03T17:56:28Z)
• Reefknot: A Comprehensive Benchmark for Relation Hallucination Evaluation, Analysis and Mitigation in Multimodal Large Language Models [13.48296910438554]
  We introduce Reefknot, a comprehensive benchmark targeting relation hallucinations, comprising over 20,000 real-world samples.<n>We provide a systematic definition of relation hallucinations, integrating perceptive and cognitive perspectives, and construct a relation-based corpus using the Visual Genome scene graph dataset.<n>We propose a novel confidence-based mitigation strategy, which reduces the hallucination rate by an average of 9.75% across three datasets, including Reefknot.
  arXiv  Detail & Related papers  (2024-08-18T10:07:02Z)
• Evaluating and Analyzing Relationship Hallucinations in Large Vision-Language Models [69.79709804046325]
  We introduce R-Bench, a novel benchmark for evaluating Vision Relationship Hallucination. R-Bench features image-level questions that focus on the existence of relationships and instance-level questions that assess local visual comprehension. We identify three types of relationship co-occurrences that lead to hallucinations: relationship-relationship, subject-relationship, and relationship-object.
  arXiv  Detail & Related papers  (2024-06-24T08:42:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.