Mitigating Hallucination in Visual Language Models with Visual
Supervision
- URL: http://arxiv.org/abs/2311.16479v1
- Date: Mon, 27 Nov 2023 09:30:02 GMT
- Title: Mitigating Hallucination in Visual Language Models with Visual
Supervision
- Authors: Zhiyang Chen, Yousong Zhu, Yufei Zhan, Zhaowen Li, Chaoyang Zhao,
Jinqiao Wang, Ming Tang
- Abstract summary: Large vision-language models (LVLMs) suffer from hallucination a lot.
Key problem lies in its weak ability to comprehend detailed content in a multi-modal context.
In this paper, we bring more detailed vision annotations and more discriminative vision models to facilitate the training of LVLMs.
- Score: 33.05550629039951
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Large vision-language models (LVLMs) suffer from hallucination a lot,
generating responses that apparently contradict to the image content
occasionally. The key problem lies in its weak ability to comprehend detailed
content in a multi-modal context, which can be mainly attributed to two factors
in training data and loss function. The vision instruction dataset primarily
focuses on global description, and the auto-regressive loss function favors
text modeling rather than image understanding. In this paper, we bring more
detailed vision annotations and more discriminative vision models to facilitate
the training of LVLMs, so that they can generate more precise responses without
encounter hallucination. On one hand, we generate image-text pairs with
detailed relationship annotations in panoptic scene graph dataset (PSG). These
conversations pay more attention on detailed facts in the image, encouraging
the model to answer questions based on multi-modal contexts. On the other hand,
we integrate SAM and mask prediction loss as auxiliary supervision, forcing the
LVLMs to have the capacity to identify context-related objects, so that they
can generate more accurate responses, mitigating hallucination. Moreover, to
provide a deeper evaluation on the hallucination in LVLMs, we propose a new
benchmark, RAH-Bench. It divides vision hallucination into three different
types that contradicts the image with wrong categories, attributes or
relations, and introduces False Positive Rate as detailed sub-metric for each
type. In this benchmark, our approach demonstrates an +8.4% enhancement
compared to original LLaVA and achieves widespread performance improvements
across other models.
Related papers
- VideoHallucer: Evaluating Intrinsic and Extrinsic Hallucinations in Large Video-Language Models [59.05674402770661]
This work introduces VideoHallucer, the first comprehensive benchmark for hallucination detection in large video-language models (LVLMs)
VideoHallucer categorizes hallucinations into two main types: intrinsic and extrinsic, offering further subcategories for detailed analysis.
arXiv Detail & Related papers (2024-06-24T06:21:59Z) - AUTOHALLUSION: Automatic Generation of Hallucination Benchmarks for Vision-Language Models [91.78328878860003]
Large vision-language models (LVLMs) hallucinate: certain context cues in an image may trigger the language module's overconfident and incorrect reasoning on abnormal or hypothetical objects.
We develop the first automatic benchmark generation approach, AUTOHALLUSION, that harnesses a few principal strategies to create diverse examples.
It generates image-based questions whose ground-truth answers contradict the language module's prior.
A model has to overcome contextual biases and distractions to reach correct answers, while incorrect or inconsistent answers indicate hallucinations.
arXiv Detail & Related papers (2024-06-16T11:44:43Z) - Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding [25.489832294197797]
This paper introduces the Instruction Contrastive Decoding (ICD) method, a novel approach designed to reduce hallucinations during LVLM inference.
Our method is inspired by our observation that what we call disturbance instructions significantly exacerbate hallucinations in multimodal fusion modules.
arXiv Detail & Related papers (2024-03-27T16:04:47Z) - Multi-Modal Hallucination Control by Visual Information Grounding [121.6983694815504]
We show that Generative Vision-Language Models (VLMs) are prone to generate plausible-sounding textual answers that are not always grounded in the input image.
We introduce Multi-Modal Mutual-Information Decoding (M3ID), a new sampling method for prompt amplification.
M3ID amplifies the influence of the reference image over the language prior, hence favoring the generation of tokens with higher mutual information with the visual prompt.
arXiv Detail & Related papers (2024-03-20T22:05:18Z) - Hallucination Augmented Contrastive Learning for Multimodal Large
Language Model [53.65682783591723]
Multi-modal large language models (MLLMs) have been shown to efficiently integrate natural language with visual information to handle multi-modal tasks.
However, MLLMs still face a fundamental limitation of hallucinations, where they tend to generate erroneous or fabricated information.
In this paper, we address hallucinations in MLLMs from a novel perspective of representation learning.
arXiv Detail & Related papers (2023-12-12T04:05:15Z) - Analyzing and Mitigating Object Hallucination in Large Vision-Language Models [110.12460299261531]
Large vision-language models (LVLMs) have shown remarkable abilities in understanding visual information with human languages.
LVLMs still suffer from object hallucination, which is the problem of generating descriptions that include objects that do not actually exist in the images.
We propose a powerful algorithm, LVLM Hallucination Revisor (LURE), to rectify object hallucination in LVLMs by reconstructing less hallucinatory descriptions.
arXiv Detail & Related papers (2023-10-01T18:10:53Z) - Detecting and Preventing Hallucinations in Large Vision Language Models [4.7264116948935975]
M-HalDetect is the first multi-modal hallucination detection dataset for detailed image descriptions.
We train fine-grained multi-modal reward models from InstructBLIP and evaluate their effectiveness with best-of-n rejection sampling.
We find that our reward model generalizes to other multi-modal models, reducing hallucinations in LLaVA and mPLUG-OWL by 15% and 57% respectively.
arXiv Detail & Related papers (2023-08-11T21:35:20Z) - Plausible May Not Be Faithful: Probing Object Hallucination in
Vision-Language Pre-training [66.0036211069513]
Large-scale vision-language pre-trained models are prone to hallucinate non-existent visual objects when generating text.
We show that models achieving better scores on standard metrics could hallucinate objects more frequently.
Surprisingly, we find that patch-based features perform the best and smaller patch resolution yields a non-trivial reduction in object hallucination.
arXiv Detail & Related papers (2022-10-14T10:27:22Z) - Thinking Hallucination for Video Captioning [0.76146285961466]
In video captioning, there are two kinds of hallucination: object and action hallucination.
We identify three main factors: (i) inadequate visual features extracted from pre-trained models, (ii) improper influences of source and target contexts during multi-modal fusion, and (iii) exposure bias in the training strategy.
Our method achieves state-of-the-art performance on the MSR-Video to Text (MSR-VTT) and the Microsoft Research Video Description Corpus (MSVD) datasets.
arXiv Detail & Related papers (2022-09-28T06:15:42Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.