LLMs Can Check Their Own Results to Mitigate Hallucinations in Traffic Understanding Tasks

• URL: http://arxiv.org/abs/2409.12580v1
• Date: Thu, 19 Sep 2024 09:02:20 GMT
• Title: LLMs Can Check Their Own Results to Mitigate Hallucinations in Traffic Understanding Tasks
• Authors: Malsha Ashani Mahawatta Dona, Beatriz Cabrero-Daniel, Yinan Yu, Christian Berger,
• Abstract summary: This paper explores the adoption of SelfCheckGPT to spot hallucinations by three state-of-the-art Large Language Models (LLMs) Our results show GPT-4o is better at generating faithful image captions than LLaVA, whereas the former demonstrated leniency in mislabeling non-hallucinated content as hallucinations.
• Score: 1.701722696403793
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Today's Large Language Models (LLMs) have showcased exemplary capabilities, ranging from simple text generation to advanced image processing. Such models are currently being explored for in-vehicle services such as supporting perception tasks in Advanced Driver Assistance Systems (ADAS) or Autonomous Driving (AD) systems, given the LLMs' capabilities to process multi-modal data. However, LLMs often generate nonsensical or unfaithful information, known as ``hallucinations'': a notable issue that needs to be mitigated. In this paper, we systematically explore the adoption of SelfCheckGPT to spot hallucinations by three state-of-the-art LLMs (GPT-4o, LLaVA, and Llama3) when analysing visual automotive data from two sources: Waymo Open Dataset, from the US, and PREPER CITY dataset, from Sweden. Our results show that GPT-4o is better at generating faithful image captions than LLaVA, whereas the former demonstrated leniency in mislabeling non-hallucinated content as hallucinations compared to the latter. Furthermore, the analysis of the performance metrics revealed that the dataset type (Waymo or PREPER CITY) did not significantly affect the quality of the captions or the effectiveness of hallucination detection. However, the models showed better performance rates over images captured during daytime, compared to during dawn, dusk or night. Overall, the results show that SelfCheckGPT and its adaptation can be used to filter hallucinations in generated traffic-related image captions for state-of-the-art LLMs.

Related papers

• Mitigating Hallucinations in Multimodal LLMs via Object-aware Preference Optimization [55.543583937522804]
  Multimodal Large Language Models (MLLMs) emerge as a unified interface to address a multitude of tasks.<n>Despite showcasing state-of-the-art results in many benchmarks, a long-standing issue is the tendency of MLLMs to hallucinate.<n>In this paper, we address the problem of hallucinations as an alignment problem, seeking to steer the MLLM so that it prefers generating content without hallucinations.
  arXiv  Detail & Related papers  (2025-08-27T18:02:04Z)
• Mitigating Object Hallucination via Robust Local Perception Search [11.570368427723961]
  Local Perception Search (LPS) is a decoding method during inference that is both simple and training-free, yet effectively suppresses hallucinations.<n>We show that LPS significantly reduces the incidence of hallucinations compared to the baseline, showing exceptional performance, particularly in noisy settings.
  arXiv  Detail & Related papers  (2025-06-07T09:27:26Z)
• Mitigating Hallucination in VideoLLMs via Temporal-Aware Activation Engineering [83.63437999696954]
  hallucination in large language models (MLLMs) persists as a significant and under-addressed challenge in the video domain.<n>We propose a temporal-aware activation engineering framework for VideoLLMs, which adaptively identifies and manipulates hallucination-sensitive modules.
  arXiv  Detail & Related papers  (2025-05-19T08:12:06Z)
• Can LLMs Detect Intrinsic Hallucinations in Paraphrasing and Machine Translation? [7.416552590139255]
  We evaluate a suite of open-access LLMs on their ability to detect intrinsic hallucinations in two conditional generation tasks. We study how model performance varies across tasks and language. We find that performance varies across models but is consistent across prompts.
  arXiv  Detail & Related papers  (2025-04-29T12:30:05Z)
• A Survey of Hallucination in Large Visual Language Models [48.794850395309076]
  The existence of hallucinations has limited the potential and practical effectiveness of LVLM in various fields. The structure of LVLMs and main causes of hallucination generation are introduced. The available hallucination evaluation benchmarks for LVLMs are presented.
  arXiv  Detail & Related papers  (2024-10-20T10:58:58Z)
• MLLM can see? Dynamic Correction Decoding for Hallucination Mitigation [50.73561815838431]
  Multimodal Large Language Models (MLLMs) frequently exhibit hallucination phenomena. We propose a novel dynamic correction decoding method for MLLMs (DeCo) We evaluate DeCo on widely-used benchmarks, demonstrating that it can reduce hallucination rates by a large margin compared to baselines.
  arXiv  Detail & Related papers  (2024-10-15T16:57:44Z)
• Evaluating and Enhancing Trustworthiness of LLMs in Perception Tasks [1.701722696403793]
  multimodal Large Language Models (LLMs) are capable of processing text, images, audio, and other data types. This paper systematically assessed potential hallucination detection strategies for such LLMs. We evaluate and extend hallucination detection approaches when applying these LLMs to video sequences in the example of pedestrian detection.
  arXiv  Detail & Related papers  (2024-07-18T20:58:03Z)
• Mitigating Object Hallucination in MLLMs via Data-augmented Phrase-level Alignment [52.43197107069751]
  Multimodal Large Language Models (MLLMs) often generate factually inaccurate information, referred to as hallucination. We introduce Data-augmented Phrase-level Alignment (DPA), a novel loss which can be applied to instruction-tuned off-the-shelf MLLMs to mitigate hallucinations.
  arXiv  Detail & Related papers  (2024-05-28T23:36:00Z)
• PoLLMgraph: Unraveling Hallucinations in Large Language Models via State Transition Dynamics [51.17512229589]
  PoLLMgraph is a model-based white-box detection and forecasting approach for large language models. We show that hallucination can be effectively detected by analyzing the LLM's internal state transition dynamics. Our work paves a new way for model-based white-box analysis of LLMs, motivating the research community to further explore, understand, and refine the intricate dynamics of LLM behaviors.
  arXiv  Detail & Related papers  (2024-04-06T20:02:20Z)
• Aligning Modalities in Vision Large Language Models via Preference Fine-tuning [67.62925151837675]
  In this work, we frame the hallucination problem as an alignment issue, tackle it with preference tuning. Specifically, we propose POVID to generate feedback data with AI models. We use ground-truth instructions as the preferred response and a two-stage approach to generate dispreferred data. In experiments across broad benchmarks, we show that we can not only reduce hallucinations, but improve model performance across standard benchmarks, outperforming prior approaches.
  arXiv  Detail & Related papers  (2024-02-18T00:56:16Z)
• Mitigating Object Hallucination in Large Vision-Language Models via Image-Grounded Guidance [51.30560006045442]
  Image-gRounded guIdaNcE (MARINE) is a framework that is both training-free and API-free.<n>MARINE effectively and efficiently reduces object hallucinations during inference by introducing image-grounded guidance to LVLMs.<n>Our framework's flexibility further allows for the integration of multiple vision models, enabling more reliable and robust object-level guidance.
  arXiv  Detail & Related papers  (2024-02-13T18:59:05Z)
• Large Language Models are Null-Shot Learners [1.424005404275135]
  Null-shot prompting exploits hallucination in large language models (LLMs) We show that it is possible to exploit hallucination to increase performance in performing tasks compared to standard zero-shot prompting.
  arXiv  Detail & Related papers  (2024-01-16T10:53:11Z)
• AutoHall: Automated Hallucination Dataset Generation for Large Language Models [56.92068213969036]
  This paper introduces a method for automatically constructing model-specific hallucination datasets based on existing fact-checking datasets called AutoHall. We also propose a zero-resource and black-box hallucination detection method based on self-contradiction.
  arXiv  Detail & Related papers  (2023-09-30T05:20:02Z)
• CIEM: Contrastive Instruction Evaluation Method for Better Instruction Tuning [8.217445461627797]
  Vision-Language Models (VLMs) may generate incorrect perception information when doing downstream applications, for example, captioning a non-existent entity. To address the hallucination phenomenon, we introduce a Contrastive Instruction Evaluation Method (CIEM) and Contrastive Instruction Tuning (CIT) We pinpoint the hallucination issues commonly present in existing VLMs, the disability of the current instruction-tuning dataset to handle the hallucination phenomenon and the superiority of CIT-tuned VLMs over both CIEM and public datasets.
  arXiv  Detail & Related papers  (2023-09-05T15:06:37Z)
• Evaluating Object Hallucination in Large Vision-Language Models [122.40337582958453]
  This work presents the first systematic study on object hallucination of large vision-language models (LVLMs) We find that LVLMs tend to generate objects that are inconsistent with the target images in the descriptions. We propose a polling-based query method called POPE to evaluate the object hallucination.
  arXiv  Detail & Related papers  (2023-05-17T16:34:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.