Pre-Training Multimodal Hallucination Detectors with Corrupted Grounding Data

• URL: http://arxiv.org/abs/2409.00238v1
• Date: Fri, 30 Aug 2024 20:11:00 GMT
• Title: Pre-Training Multimodal Hallucination Detectors with Corrupted Grounding Data
• Authors: Spencer Whitehead, Jacob Phillips, Sean Hendryx,
• Abstract summary: Multimodal language models can exhibit hallucinations in their outputs, which limits their reliability. We propose an approach to improve the sample efficiency of these models by creating corrupted grounding data.
• Score: 4.636499986218049
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal language models can exhibit hallucinations in their outputs, which limits their reliability. The ability to automatically detect these errors is important for mitigating them, but has been less explored and existing efforts do not localize hallucinations, instead framing this as a classification task. In this work, we first pose multimodal hallucination detection as a sequence labeling task where models must localize hallucinated text spans and present a strong baseline model. Given the high cost of human annotations for this task, we propose an approach to improve the sample efficiency of these models by creating corrupted grounding data, which we use for pre-training. Leveraging phrase grounding data, we generate hallucinations to replace grounded spans and create hallucinated text. Experiments show that pre-training on this data improves sample efficiency when fine-tuning, and that the learning signal from the grounding data plays an important role in these improvements.

Related papers

• Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedback [48.065569871444275]
  We propose detecting and mitigating hallucinations in Large Vision Language Models (LVLMs) via fine-grained AI feedback. We generate a small-size hallucination annotation dataset by proprietary models. Then, we propose a detect-then-rewrite pipeline to automatically construct preference dataset for training hallucination mitigating model.
  arXiv  Detail & Related papers  (2024-04-22T14:46:10Z)
• HalluciDoctor: Mitigating Hallucinatory Toxicity in Visual Instruction Data [102.56792377624927]
  hallucinations inherent in machine-generated data remain under-explored. We present a novel hallucination detection and elimination framework, HalluciDoctor, based on the cross-checking paradigm. Our method successfully mitigates 44.6% hallucinations relatively and maintains competitive performance compared to LLaVA.
  arXiv  Detail & Related papers  (2023-11-22T04:52:58Z)
• A New Benchmark and Reverse Validation Method for Passage-level Hallucination Detection [63.56136319976554]
  Large Language Models (LLMs) generate hallucinations, which can cause significant damage when deployed for mission-critical tasks. We propose a self-check approach based on reverse validation to detect factual errors automatically in a zero-resource fashion. We empirically evaluate our method and existing zero-resource detection methods on two datasets.
  arXiv  Detail & Related papers  (2023-10-10T10:14:59Z)
• AutoHall: Automated Hallucination Dataset Generation for Large Language Models [56.92068213969036]
  This paper introduces a method for automatically constructing model-specific hallucination datasets based on existing fact-checking datasets called AutoHall. We also propose a zero-resource and black-box hallucination detection method based on self-contradiction.
  arXiv  Detail & Related papers  (2023-09-30T05:20:02Z)
• Reducing Hallucinations in Neural Machine Translation with Feature Attribution [54.46113444757899]
  We present a case study focusing on model understanding and regularisation to reduce hallucinations in NMT. We first use feature attribution methods to study the behaviour of an NMT model that produces hallucinations. We then leverage these methods to propose a novel loss function that substantially helps reduce hallucinations and does not require retraining the model from scratch.
  arXiv  Detail & Related papers  (2022-11-17T20:33:56Z)
• Looking for a Needle in a Haystack: A Comprehensive Study of Hallucinations in Neural Machine Translation [17.102338932907294]
  We set foundations for the study of NMT hallucinations. We propose DeHallucinator, a simple method for alleviating hallucinations at test time.
  arXiv  Detail & Related papers  (2022-08-10T12:44:13Z)
• Can a Transformer Pass the Wug Test? Tuning Copying Bias in Neural Morphological Inflection Models [9.95909045828344]
  We show that, to be more effective, the hallucination process needs to pay attention to syllable-like length rather than individual characters or stems. We report a significant performance improvement with our hallucination model over previous data hallucination methods when training and test data do not overlap in their lemmata.
  arXiv  Detail & Related papers  (2021-04-13T19:51:21Z)
• Detecting Hallucinated Content in Conditional Neural Sequence Generation [165.68948078624499]
  We propose a task to predict whether each token in the output sequence is hallucinated (not contained in the input) We also introduce a method for learning to detect hallucinations using pretrained language models fine tuned on synthetic data.
  arXiv  Detail & Related papers  (2020-11-05T00:18:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.