ODE: Open-Set Evaluation of Hallucinations in Multimodal Large Language Models

• URL: http://arxiv.org/abs/2409.09318v1
• Date: Sat, 14 Sep 2024 05:31:29 GMT
• Title: ODE: Open-Set Evaluation of Hallucinations in Multimodal Large Language Models
• Authors: Yahan Tu, Rui Hu, Jitao Sang,
• Abstract summary: This paper introduces ODE, an open-set, dynamic protocol for evaluating object existence hallucinations in large language models (MLLMs) Our framework employs graph structures to model associations between real-word concepts and generates novel samples for both general and domain-specific scenarios. Experimental results show that MLLMs exhibit higher hallucination rates with ODE-generated samples, effectively avoiding data contamination.
• Score: 15.156359255401812
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Hallucination poses a significant challenge for multimodal large language models (MLLMs). However, existing benchmarks for evaluating hallucinations are static, which can lead to potential data contamination. This paper introduces ODE, an open-set, dynamic protocol for evaluating object existence hallucinations in MLLMs. Our framework employs graph structures to model associations between real-word concepts and generates novel samples for both general and domain-specific scenarios. The dynamic combination of concepts, along with various combination principles, ensures a broad sample distribution. Experimental results show that MLLMs exhibit higher hallucination rates with ODE-generated samples, effectively avoiding data contamination. Moreover, these samples can also be used for fine-tuning to improve MLLM performance on existing benchmarks.

Related papers

• Efficient Contrastive Decoding with Probabilistic Hallucination Detection - Mitigating Hallucinations in Large Vision Language Models - [1.2499537119440245]
  Efficient Contrastive Decoding (ECD) is a simple method that leverages probabilistic hallucination detection to shift the output distribution towards contextually accurate answers at inference time. Our experiments show that ECD effectively mitigates hallucinations, outperforming state-of-the-art methods with respect to performance on LVLM benchmarks and computation time.
  arXiv  Detail & Related papers  (2025-04-16T14:50:25Z)
• Enhancing Hallucination Detection through Noise Injection [9.582929634879932]
  Large Language Models (LLMs) are prone to generating plausible yet incorrect responses, known as hallucinations. We show that detection can be improved significantly by taking into account model uncertainty in the Bayesian sense. We propose a very simple and efficient approach that perturbs an appropriate subset of model parameters, or equivalently hidden unit activations, during sampling.
  arXiv  Detail & Related papers  (2025-02-06T06:02:20Z)
• MLLM can see? Dynamic Correction Decoding for Hallucination Mitigation [50.73561815838431]
  Multimodal Large Language Models (MLLMs) frequently exhibit hallucination phenomena. We propose a novel dynamic correction decoding method for MLLMs (DeCo) We evaluate DeCo on widely-used benchmarks, demonstrating that it can reduce hallucination rates by a large margin compared to baselines.
  arXiv  Detail & Related papers  (2024-10-15T16:57:44Z)
• OAEI-LLM: A Benchmark Dataset for Understanding Large Language Model Hallucinations in Ontology Matching [8.732396482276332]
  Hallucinations of large language models (LLMs) commonly occur in domain-specific downstream tasks, with no exception in ontology matching (OM) The OAEI-LLM dataset is an extended version of the Ontology Alignment Evaluation Initiative (OAEI) datasets that evaluate LLM-specific hallucinations in OM tasks.
  arXiv  Detail & Related papers  (2024-09-21T06:49:34Z)
• Improving Factuality in Large Language Models via Decoding-Time Hallucinatory and Truthful Comparators [14.705475420665117]
  Large Language Models (LLMs) are prone to generate responses that contradict verifiable facts. We propose a Comparator-driven Decoding-Time (CDT) framework to alleviate the response hallucination.
  arXiv  Detail & Related papers  (2024-08-22T12:00:31Z)
• CODE: Contrasting Self-generated Description to Combat Hallucination in Large Multi-modal Models [51.70129969269271]
  We introduce a novel contrastive-based decoding method, COuntering DEscription Contrastive Decoding (CODE) Our method significantly reduces hallucinations and improves cross-modal consistency across various benchmarks and cutting-edge LMMs.
  arXiv  Detail & Related papers  (2024-06-04T03:04:21Z)
• VALOR-EVAL: Holistic Coverage and Faithfulness Evaluation of Large Vision-Language Models [57.43276586087863]
  Large Vision-Language Models (LVLMs) suffer from hallucination issues, wherein the models generate plausible-sounding but factually incorrect outputs. Existing benchmarks are often limited in scope, focusing mainly on object hallucinations. We introduce a multi-dimensional benchmark covering objects, attributes, and relations, with challenging images selected based on associative biases.
  arXiv  Detail & Related papers  (2024-04-22T04:49:22Z)
• Prescribing the Right Remedy: Mitigating Hallucinations in Large Vision-Language Models via Targeted Instruction Tuning [15.156359255401812]
  We propose a targeted instruction data generation framework named DFTG that tailored to the hallucination specificity of different models. The experimental results on hallucination benchmarks demonstrate that the targeted instruction data generated by our method are more effective in mitigating hallucinations compared to previous datasets.
  arXiv  Detail & Related papers  (2024-04-16T07:14:32Z)
• PoLLMgraph: Unraveling Hallucinations in Large Language Models via State Transition Dynamics [51.17512229589]
  PoLLMgraph is a model-based white-box detection and forecasting approach for large language models. We show that hallucination can be effectively detected by analyzing the LLM's internal state transition dynamics. Our work paves a new way for model-based white-box analysis of LLMs, motivating the research community to further explore, understand, and refine the intricate dynamics of LLM behaviors.
  arXiv  Detail & Related papers  (2024-04-06T20:02:20Z)
• Dual-View Data Hallucination with Semantic Relation Guidance for Few-Shot Image Recognition [49.26065739704278]
  We propose a framework that exploits semantic relations to guide dual-view data hallucination for few-shot image recognition. An instance-view data hallucination module hallucinates each sample of a novel class to generate new data. A prototype-view data hallucination module exploits semantic-aware measure to estimate the prototype of a novel class.
  arXiv  Detail & Related papers  (2024-01-13T12:32:29Z)
• RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models [9.465753274663061]
  Retrieval-augmented generation (RAG) has become a main technique for alleviating hallucinations in large language models (LLMs) This paper presents RAGTruth, a corpus tailored for analyzing word-level hallucinations in various domains.
  arXiv  Detail & Related papers  (2023-12-31T04:43:45Z)
• AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation [58.19101663976327]
  Multi-modal Large Language Models (MLLMs) encounter the significant challenge of hallucinations. evaluating MLLMs' hallucinations is becoming increasingly important in model improvement and practical application deployment. We propose an LLM-free multi-dimensional benchmark AMBER, which can be used to evaluate both generative task and discriminative task.
  arXiv  Detail & Related papers  (2023-11-13T15:25:42Z)
• ReEval: Automatic Hallucination Evaluation for Retrieval-Augmented Large Language Models via Transferable Adversarial Attacks [91.55895047448249]
  This paper presents ReEval, an LLM-based framework using prompt chaining to perturb the original evidence for generating new test cases. We implement ReEval using ChatGPT and evaluate the resulting variants of two popular open-domain QA datasets. Our generated data is human-readable and useful to trigger hallucination in large language models.
  arXiv  Detail & Related papers  (2023-10-19T06:37:32Z)
• AutoHall: Automated Hallucination Dataset Generation for Large Language Models [56.92068213969036]
  This paper introduces a method for automatically constructing model-specific hallucination datasets based on existing fact-checking datasets called AutoHall. We also propose a zero-resource and black-box hallucination detection method based on self-contradiction.
  arXiv  Detail & Related papers  (2023-09-30T05:20:02Z)
• Detecting and Preventing Hallucinations in Large Vision Language Models [4.7264116948935975]
  M-HalDetect is the first multi-modal hallucination detection dataset for detailed image descriptions. We train fine-grained multi-modal reward models from InstructBLIP and evaluate their effectiveness with best-of-n rejection sampling. We find that our reward model generalizes to other multi-modal models, reducing hallucinations in LLaVA and mPLUG-OWL by 15% and 57% respectively.
  arXiv  Detail & Related papers  (2023-08-11T21:35:20Z)
• Integrating Expert ODEs into Neural ODEs: Pharmacology and Disease Progression [71.7560927415706]
  latent hybridisation model (LHM) integrates a system of expert-designed ODEs with machine-learned Neural ODEs to fully describe the dynamics of the system. We evaluate LHM on synthetic data as well as real-world intensive care data of COVID-19 patients.
  arXiv  Detail & Related papers  (2021-06-05T11:42:45Z)
• Robust Finite Mixture Regression for Heterogeneous Targets [70.19798470463378]
  We propose an FMR model that finds sample clusters and jointly models multiple incomplete mixed-type targets simultaneously. We provide non-asymptotic oracle performance bounds for our model under a high-dimensional learning framework. The results show that our model can achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-10-12T03:27:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.