Mitigating Knowledge Conflicts in Language Model-Driven Question Answering

• URL: http://arxiv.org/abs/2411.11344v1
• Date: Mon, 18 Nov 2024 07:33:10 GMT
• Title: Mitigating Knowledge Conflicts in Language Model-Driven Question Answering
• Authors: Han Cao, Zhaoyang Zhang, Xiangtian Li, Chufan Wu, Hansong Zhang, Wenqing Zhang,
• Abstract summary: In this work, we argue that hallucination could be mitigated via explicit correlation between input source and generated content. We focus on a typical example of hallucination, entity-based knowledge conflicts in question answering, where correlation of entities and their description at training time hinders model behaviour during inference.
• Score: 15.29366851382021
• License:
• Abstract: Knowledge-aware sequence to sequence generation tasks such as document question answering and abstract summarization typically requires two types of knowledge: encoded parametric knowledge and retrieved contextual information. Previous work show improper correlation between parametric knowledge and answers in the training set could cause the model ignore input information at test time, resulting in un-desirable model behaviour such as over-stability and hallucination. In this work, we argue that hallucination could be mitigated via explicit correlation between input source and generated content. We focus on a typical example of hallucination, entity-based knowledge conflicts in question answering, where correlation of entities and their description at training time hinders model behaviour during inference.

Related papers

• Distinguishing Ignorance from Error in LLM Hallucinations [43.62904897907926]
  We focus on close-book Question Answering (CBQA), where previous work has not fully addressed the distinction between two possible kinds of hallucinations. We argue that distinguishing these cases is crucial for detecting and mitigating hallucinations.
  arXiv  Detail & Related papers  (2024-10-29T14:31:33Z)
• Studying Large Language Model Behaviors Under Context-Memory Conflicts With Real Documents [54.953320616069654]
  Retrieval-augmented generation mitigates many problems of fully parametric language models. In RAG, the model's knowledge can be updated from documents provided in context. We present a framework for studying such knowledge conflicts in a realistic setup.
  arXiv  Detail & Related papers  (2024-04-24T17:59:36Z)
• Does the Generator Mind its Contexts? An Analysis of Generative Model Faithfulness under Context Transfer [42.081311699224585]
  The present study introduces the knowledge-augmented generator, which is specifically designed to produce information that remains grounded in contextual knowledge. Our objective is to explore the existence of hallucinations arising from parametric memory when contextual knowledge undergoes changes.
  arXiv  Detail & Related papers  (2024-02-22T12:26:07Z)
• Blending Reward Functions via Few Expert Demonstrations for Faithful and Accurate Knowledge-Grounded Dialogue Generation [22.38338205905379]
  We leverage reinforcement learning algorithms to overcome the above challenges by introducing a novel reward function. Our reward function combines an accuracy metric and a faithfulness metric to provide a balanced quality judgment of generated responses.
  arXiv  Detail & Related papers  (2023-11-02T02:42:41Z)
• Towards Mitigating Hallucination in Large Language Models via Self-Reflection [63.2543947174318]
  Large language models (LLMs) have shown promise for generative and knowledge-intensive tasks including question-answering (QA) tasks. This paper analyses the phenomenon of hallucination in medical generative QA systems using widely adopted LLMs and datasets.
  arXiv  Detail & Related papers  (2023-10-10T03:05:44Z)
• Getting Sick After Seeing a Doctor? Diagnosing and Mitigating Knowledge Conflicts in Event Temporal Reasoning [87.92209048521153]
  Event temporal reasoning aims at identifying the temporal relations between two or more events from narratives. Knowledge conflicts arise when there is a mismatch between the actual temporal relations of events in the context and the prior knowledge or biases learned by the model.
  arXiv  Detail & Related papers  (2023-05-24T10:04:06Z)
• RECKONING: Reasoning through Dynamic Knowledge Encoding [51.076603338764706]
  We show that language models can answer questions by reasoning over knowledge provided as part of the context. In these situations, the model fails to distinguish the knowledge that is necessary to answer the question. We propose teaching the model to reason more robustly by folding the provided contextual knowledge into the model's parameters.
  arXiv  Detail & Related papers  (2023-05-10T17:54:51Z)
• The KITMUS Test: Evaluating Knowledge Integration from Multiple Sources in Natural Language Understanding Systems [87.3207729953778]
  We evaluate state-of-the-art coreference resolution models on our dataset. Several models struggle to reason on-the-fly over knowledge observed both at pretrain time and at inference time. Still, even the best performing models seem to have difficulties with reliably integrating knowledge presented only at inference time.
  arXiv  Detail & Related papers  (2022-12-15T23:26:54Z)
• Entity-Based Knowledge Conflicts in Question Answering [29.973926661540524]
  We formalize the problem of knowledge conflicts, where the contextual information contradicts the learned information. We propose a method to mitigate over-reliance on parametric knowledge, which minimizes hallucination, and improves out-of-distribution generalization by 4%-7%. Our findings demonstrate the importance for practitioners to evaluate model tendency to hallucinate rather than read, and show that our mitigation strategy encourages generalization to evolving information.
  arXiv  Detail & Related papers  (2021-09-10T18:29:44Z)
• Visual Question Answering with Prior Class Semantics [50.845003775809836]
  We show how to exploit additional information pertaining to the semantics of candidate answers. We extend the answer prediction process with a regression objective in a semantic space. Our method brings improvements in consistency and accuracy over a range of question types.
  arXiv  Detail & Related papers  (2020-05-04T02:46:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.