Mitigating Knowledge Conflicts in Language Model-Driven Question Answering

• URL: http://arxiv.org/abs/2411.11344v3
• Date: Wed, 15 Jan 2025 07:46:15 GMT
• Title: Mitigating Knowledge Conflicts in Language Model-Driven Question Answering
• Authors: Han Cao, Zhaoyang Zhang, Xiangtian Li, Chufan Wu, Hansong Zhang, Wenqing Zhang,
• Abstract summary: Two fundamental knowledge sources play crucial roles in document-based question answering and document summarization systems. Recent studies revealed a significant challenge: when there exists a misalignment between the model's inherent knowledge and the ground truth answers in training data, the system may exhibit problematic behaviors during inference. Our investigation proposes a strategy to minimize hallucination by building explicit connection between source inputs and generated outputs.
• Score: 15.29366851382021
• License:
• Abstract: In the context of knowledge-driven seq-to-seq generation tasks, such as document-based question answering and document summarization systems, two fundamental knowledge sources play crucial roles: the inherent knowledge embedded within model parameters and the external knowledge obtained through context. Recent studies revealed a significant challenge: when there exists a misalignment between the model's inherent knowledge and the ground truth answers in training data, the system may exhibit problematic behaviors during inference, such as ignoring input context, or generating unfaithful content. Our investigation proposes a strategy to minimize hallucination by building explicit connection between source inputs and generated outputs. We specifically target a common hallucination pattern in question answering, examining how the correspondence between entities and their contexts during model training influences the system's performance at inference time.

Related papers

• On the Loss of Context-awareness in General Instruction Fine-tuning [101.03941308894191]
  We investigate the loss of context awareness after supervised fine-tuning. We find that the performance decline is associated with a bias toward different roles learned during conversational instruction fine-tuning. We propose a metric to identify context-dependent examples from general instruction fine-tuning datasets.
  arXiv  Detail & Related papers  (2024-11-05T00:16:01Z)
• Context Matters: An Empirical Study of the Impact of Contextual Information in Temporal Question Answering Systems [7.393290178125003]
  This paper empirically examines the robustness of temporal question-answering systems trained on various context types. We show that training with a mix of these contexts enhances model robustness and accuracy. We introduce two new context-rich TQA datasets, ContextAQA and ContextTQE, and provide comprehensive evaluations and guidelines for training robust TQA models.
  arXiv  Detail & Related papers  (2024-06-27T21:31:30Z)
• Learning to Filter Context for Retrieval-Augmented Generation [75.18946584853316]
  Generation models are required to generate outputs given partially or entirely irrelevant passages. FILCO identifies useful context based on lexical and information-theoretic approaches. It trains context filtering models that can filter retrieved contexts at test time.
  arXiv  Detail & Related papers  (2023-11-14T18:41:54Z)
• DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain Question Answering over Knowledge Base and Text [73.68051228972024]
  Large Language Models (LLMs) have exhibited impressive generation capabilities, but they suffer from hallucinations when relying on their internal knowledge. Retrieval-augmented LLMs have emerged as a potential solution to ground LLMs in external knowledge.
  arXiv  Detail & Related papers  (2023-10-31T04:37:57Z)
• Contextual Knowledge Learning For Dialogue Generation [13.671946960656467]
  We present a novel approach to context and knowledge weighting as an integral part of model training. We guide the model training through a Contextual Knowledge Learning process which involves Latent Vectors for context and knowledge.
  arXiv  Detail & Related papers  (2023-05-29T16:54:10Z)
• The KITMUS Test: Evaluating Knowledge Integration from Multiple Sources in Natural Language Understanding Systems [87.3207729953778]
  We evaluate state-of-the-art coreference resolution models on our dataset. Several models struggle to reason on-the-fly over knowledge observed both at pretrain time and at inference time. Still, even the best performing models seem to have difficulties with reliably integrating knowledge presented only at inference time.
  arXiv  Detail & Related papers  (2022-12-15T23:26:54Z)
• Rich Knowledge Sources Bring Complex Knowledge Conflicts: Recalibrating Models to Reflect Conflicting Evidence [37.18100697469402]
  We simulate knowledge conflicts where parametric knowledge suggests one answer and different passages suggest different answers. We find retrieval performance heavily impacts which sources models rely on, and current models mostly rely on non-performing knowledge. We present a new calibration study, where models are discouraged from presenting any single answer when presented with multiple conflicting answer candidates.
  arXiv  Detail & Related papers  (2022-10-25T01:46:00Z)
• elBERto: Self-supervised Commonsense Learning for Question Answering [131.51059870970616]
  We propose a Self-supervised Bidirectional Representation Learning of Commonsense framework, which is compatible with off-the-shelf QA model architectures. The framework comprises five self-supervised tasks to force the model to fully exploit the additional training signals from contexts containing rich commonsense. elBERto achieves substantial improvements on out-of-paragraph and no-effect questions where simple lexical similarity comparison does not help.
  arXiv  Detail & Related papers  (2022-03-17T16:23:45Z)
• Knowledge-Grounded Dialogue Generation with a Unified Knowledge Representation [78.85622982191522]
  Existing systems perform poorly on unseen topics due to limited topics covered in the training data. We present PLUG, a language model that homogenizes different knowledge sources to a unified knowledge representation. It can achieve comparable performance with state-of-the-art methods under a fully-supervised setting.
  arXiv  Detail & Related papers  (2021-12-15T07:11:02Z)
• Zero-Resource Knowledge-Grounded Dialogue Generation [29.357221039484568]
  We propose representing the knowledge that bridges a context and a response and the way that the knowledge is expressed as latent variables. We show that our model can achieve comparable performance with state-of-the-art methods that rely on knowledge-grounded dialogues for training.
  arXiv  Detail & Related papers  (2020-08-29T05:48:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.