DAMRO: Dive into the Attention Mechanism of LVLM to Reduce Object Hallucination

• URL: http://arxiv.org/abs/2410.04514v1
• Date: Sun, 6 Oct 2024 15:12:09 GMT
• Title: DAMRO: Dive into the Attention Mechanism of LVLM to Reduce Object Hallucination
• Authors: Xuan Gong, Tianshi Ming, Xinpeng Wang, Zhihua Wei,
• Abstract summary: We find that the attention distribution of Large Language Model (LLM) decoder on image tokens is highly consistent with the visual encoder. We propose DAMRO, a novel training-free strategy that $D$ive into $A$ttention $M$echanism of LVLM.
• Score: 11.845711223575462
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite the great success of Large Vision-Language Models (LVLMs), they inevitably suffer from hallucination. As we know, both the visual encoder and the Large Language Model (LLM) decoder in LVLMs are Transformer-based, allowing the model to extract visual information and generate text outputs via attention mechanisms. We find that the attention distribution of LLM decoder on image tokens is highly consistent with the visual encoder and both distributions tend to focus on particular background tokens rather than the referred objects in the image. We attribute to the unexpected attention distribution to an inherent flaw in the visual encoder itself, which misguides LLMs to over emphasize the redundant information and generate object hallucination. To address the issue, we propose DAMRO, a novel training-free strategy that $D$ive into $A$ttention $M$echanism of LVLM to $R$educe $O$bject Hallucination. Specifically, our approach employs classification token (CLS) of ViT to filter out high-attention outlier tokens scattered in the background and then eliminate their influence during decoding stage. We evaluate our method on LVLMs including LLaVA-1.5, LLaVA-NeXT and InstructBLIP, using various benchmarks such as POPE, CHAIR, MME and GPT-4V Aided Evaluation. The results demonstrate that our approach significantly reduces the impact of these outlier tokens, thus effectively alleviating the hallucination of LVLMs. The code of our method will be released soon.

Related papers

• Looking Beyond Text: Reducing Language bias in Large Vision-Language Models via Multimodal Dual-Attention and Soft-Image Guidance [67.26434607115392]
  Large vision-language models (LVLMs) have achieved impressive results in various vision-language tasks. LVLMs suffer from hallucinations caused by language bias, leading to diminished focus on images and ineffective visual comprehension. We propose LACING to address the language bias of LVLMs with muLtimodal duAl-attention meChanIsm (MDA) aNd soft-image Guidance (IFG)
  arXiv  Detail & Related papers  (2024-11-21T16:33:30Z)
• Mitigating Object Hallucination via Concentric Causal Attention [71.27325347912823]
  We show that object hallucination is closely tied with Rotary Position. RoPE, a widely adopted positional dependency modeling design. We propose Concentric Causal Attention (CCA), a simple yet effective positional alignment strategy.
  arXiv  Detail & Related papers  (2024-10-21T11:54:53Z)
• From Pixels to Tokens: Revisiting Object Hallucinations in Large Vision-Language Models [15.401221354325672]
  Hallucinations in large vision models (LVLMs) are a significant challenge, i.e., generating objects that are not presented in the visual input. Recent studies often attribute hallucinations to a lack of understanding of visual input, yet ignore a more fundamental issue: the model's inability to extract or decouple visual features. In this paper, we revisit the hallucinations in LVLMs from an architectural perspective, investigating whether the primary cause lies in the visual encoder (feature extraction) or the modal alignment module (feature decoupling)
  arXiv  Detail & Related papers  (2024-10-09T11:46:32Z)
• VideoLLM-MoD: Efficient Video-Language Streaming with Mixture-of-Depths Vision Computation [66.00245701441547]
  We introduce a novel approach to reduce vision compute by leveraging redundant vision tokens "skipping layers" rather than decreasing the number of vision tokens. Our method, VideoLLM-MoD, is inspired by mixture-of-depths LLMs and addresses the challenge of numerous vision tokens in long-term or streaming video.
  arXiv  Detail & Related papers  (2024-08-29T17:21:58Z)
• Mitigating Hallucinations in Large Vision-Language Models (LVLMs) via Language-Contrastive Decoding (LCD) [13.430637580980164]
  Large Vision-Language Models (LVLMs) are an extension of Large Language Models (LLMs) that facilitate processing both image and text inputs, expanding AI capabilities. Our study introduces a Language Contrastive Decoding (LCD) algorithm that adjusts LVLM outputs based on Large Language Models distribution confidence levels. Our method effectively improves LVLMs without needing complex post-processing or retraining, and is easily applicable to different models.
  arXiv  Detail & Related papers  (2024-08-06T08:10:34Z)
• Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Models [30.26685485474035]
  Large Vision-Language Models (LVLMs) have rapidly advanced in recent years. The prevalent issue known as the hallucination' problem has emerged as a significant bottleneck. We propose a simple yet effective method named Self-Introspective Decoding (SID)
  arXiv  Detail & Related papers  (2024-08-04T13:50:17Z)
• Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs [14.381188702947949]
  Large Vision-Language Models (LVLMs) primarily align image features of vision encoder with Large Language Models (LLMs) to leverage their superior text generation capabilities. This imbalance in LVLMs may result in the instances of hallucinatory. We introduce a training-free algorithm to find an equilibrium point between image comprehension and language inference.
  arXiv  Detail & Related papers  (2024-07-31T17:46:57Z)
• MetaToken: Detecting Hallucination in Image Descriptions by Meta Classification [1.3654846342364308]
  We introduce MetaToken, a lightweight binary classifier to detect hallucinations on the token-level at negligible cost. Based on a statistical analysis, we reveal key factors of hallucinations in LVLMs which have been overseen in previous works. We evaluate our method on four state-of-the-art LVLMs demonstrating the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-05-29T15:28:42Z)
• Mitigating Object Hallucination in Large Vision-Language Models via Classifier-Free Guidance [56.04768229686853]
  Large Vision-Language Models (LVLMs) tend to hallucinate non-existing objects in the images. We introduce a framework called Mitigating hallucinAtion via classifieR-Free guIdaNcE (MARINE) MARINE is both training-free and API-free, and can effectively and efficiently reduce object hallucinations during the generation process.
  arXiv  Detail & Related papers  (2024-02-13T18:59:05Z)
• OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation [124.9008419182485]
  We present OPERA, a novel MLLM decoding method grounded in an Over-trust Penalty and a Retrospection-Allocation strategy. Our approach begins with an interesting observation that, most hallucinations are closely tied to the knowledge aggregation patterns in the self-attention matrix. Based on the observation, OPERA introduces a penalty term on the model logits during the beam-search decoding to mitigate the over-trust issue.
  arXiv  Detail & Related papers  (2023-11-29T18:57:07Z)
• Evaluating Object Hallucination in Large Vision-Language Models [122.40337582958453]
  This work presents the first systematic study on object hallucination of large vision-language models (LVLMs) We find that LVLMs tend to generate objects that are inconsistent with the target images in the descriptions. We propose a polling-based query method called POPE to evaluate the object hallucination.
  arXiv  Detail & Related papers  (2023-05-17T16:34:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.