Zero-resource Hallucination Detection for Text Generation via Graph-based Contextual Knowledge Triples Modeling

• URL: http://arxiv.org/abs/2409.11283v3
• Date: Wed, 25 Sep 2024 01:55:29 GMT
• Title: Zero-resource Hallucination Detection for Text Generation via Graph-based Contextual Knowledge Triples Modeling
• Authors: Xinyue Fang, Zhen Huang, Zhiliang Tian, Minghui Fang, Ziyi Pan, Quntian Fang, Zhihua Wen, Hengyue Pan, Dongsheng Li,
• Abstract summary: Hallucination detections for text generation with open-ended answers are more challenging. Recent studies on detecting hallucinations in long text without external resources conduct consistency comparison. We propose a graph-based context-aware (GCA) hallucination detection for text generations.
• Score: 22.107187408777726
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: LLMs obtain remarkable performance but suffer from hallucinations. Most research on detecting hallucination focuses on the questions with short and concrete correct answers that are easy to check the faithfulness. Hallucination detections for text generation with open-ended answers are more challenging. Some researchers use external knowledge to detect hallucinations in generated texts, but external resources for specific scenarios are hard to access. Recent studies on detecting hallucinations in long text without external resources conduct consistency comparison among multiple sampled outputs. To handle long texts, researchers split long texts into multiple facts and individually compare the consistency of each pairs of facts. However, these methods (1) hardly achieve alignment among multiple facts; (2) overlook dependencies between multiple contextual facts. In this paper, we propose a graph-based context-aware (GCA) hallucination detection for text generations, which aligns knowledge facts and considers the dependencies between contextual knowledge triples in consistency comparison. Particularly, to align multiple facts, we conduct a triple-oriented response segmentation to extract multiple knowledge triples. To model dependencies among contextual knowledge triple (facts), we construct contextual triple into a graph and enhance triples' interactions via message passing and aggregating via RGCN. To avoid the omission of knowledge triples in long text, we conduct a LLM-based reverse verification via reconstructing the knowledge triples. Experiments show that our model enhances hallucination detection and excels all baselines.

Related papers

• Coarse-to-Fine Highlighting: Reducing Knowledge Hallucination in Large Language Models [58.952782707682815]
  COFT is a novel method to focus on different-level key texts, thereby avoiding getting lost in lengthy contexts. Experiments on the knowledge hallucination benchmark demonstrate the effectiveness of COFT, leading to a superior performance over $30%$ in the F1 score metric.
  arXiv  Detail & Related papers  (2024-10-19T13:59:48Z)
• A Multiple-Fill-in-the-Blank Exam Approach for Enhancing Zero-Resource Hallucination Detection in Large Language Models [0.9217021281095907]
  Large language models (LLMs) often fabricate a hallucinatory text. We propose a hallucination detection method that incorporates a multiple-fill-in-the-blank exam approach.
  arXiv  Detail & Related papers  (2024-09-20T04:34:30Z)
• Reefknot: A Comprehensive Benchmark for Relation Hallucination Evaluation, Analysis and Mitigation in Multimodal Large Language Models [13.48296910438554]
  Hallucination issues persistently plagued current multimodal large language models (MLLMs) We introduce Reefknot, a benchmark specifically targeting relation hallucinations, consisting of over 20,000 samples derived from real-world scenarios. Our comparative evaluation across three distinct tasks revealed a substantial shortcoming in the capabilities of current MLLMs to mitigate relation hallucinations.
  arXiv  Detail & Related papers  (2024-08-18T10:07:02Z)
• Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps [48.58310785625051]
  Large language models (LLMs) can hallucinate details and respond with unsubstantiated answers. This paper describes a simple approach for detecting such contextual hallucinations.
  arXiv  Detail & Related papers  (2024-07-09T17:44:34Z)
• Contextual Knowledge Pursuit for Faithful Visual Synthesis [33.191847768674826]
  In large language models (LLMs), a prevalent strategy to reduce hallucinations is to retrieve factual knowledge from an external database. This paper proposes Conparametric Knowledge Pursuit (CKPT), a framework that leverages the complementary strengths of external and parametric knowledge to help generators produce reliable visual content.
  arXiv  Detail & Related papers  (2023-11-29T18:51:46Z)
• Enhancing Uncertainty-Based Hallucination Detection with Stronger Focus [99.33091772494751]
  Large Language Models (LLMs) have gained significant popularity for their impressive performance across diverse fields. LLMs are prone to hallucinate untruthful or nonsensical outputs that fail to meet user expectations. We propose a novel reference-free, uncertainty-based method for detecting hallucinations in LLMs.
  arXiv  Detail & Related papers  (2023-11-22T08:39:17Z)
• DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain Question Answering over Knowledge Base and Text [73.68051228972024]
  Large Language Models (LLMs) have exhibited impressive generation capabilities, but they suffer from hallucinations when relying on their internal knowledge. Retrieval-augmented LLMs have emerged as a potential solution to ground LLMs in external knowledge.
  arXiv  Detail & Related papers  (2023-10-31T04:37:57Z)
• SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models [55.60306377044225]
  "SelfCheckGPT" is a simple sampling-based approach to fact-check the responses of black-box models. We investigate this approach by using GPT-3 to generate passages about individuals from the WikiBio dataset.
  arXiv  Detail & Related papers  (2023-03-15T19:31:21Z)
• Don't Say What You Don't Know: Improving the Consistency of Abstractive Summarization by Constraining Beam Search [54.286450484332505]
  We analyze the connection between hallucinations and training data, and find evidence that models hallucinate because they train on target summaries that are unsupported by the source. We present PINOCCHIO, a new decoding method that improves the consistency of a transformer-based abstractive summarizer by constraining beam search to avoid hallucinations.
  arXiv  Detail & Related papers  (2022-03-16T07:13:52Z)
• Commonsense Knowledge Mining from Term Definitions [0.20305676256390934]
  We investigate a few machine learning approaches to mining commonsense knowledge triples using dictionary term definitions as inputs. Our experiments show that term definitions contain some valid and novel commonsense knowledge triples for some semantic relations.
  arXiv  Detail & Related papers  (2021-02-01T05:54:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.