Resolving Knowledge Conflicts in Large Language Models
- URL: http://arxiv.org/abs/2310.00935v1
- Date: Mon, 2 Oct 2023 06:57:45 GMT
- Title: Resolving Knowledge Conflicts in Large Language Models
- Authors: Yike Wang, Shangbin Feng, Heng Wang, Weijia Shi, Vidhisha
Balachandran, Tianxing He, Yulia Tsvetkov
- Abstract summary: Large language models (LLMs) often encounter knowledge conflicts.
We ask what are the desiderata for LLMs when a knowledge conflict arises and whether existing LLMs fulfill them.
We introduce KNOWLEDGE CONFLICT, an evaluation framework for simulating contextual knowledge conflicts.
- Score: 48.92369530853202
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Large language models (LLMs) often encounter knowledge conflicts, scenarios
where discrepancy arises between the internal parametric knowledge of LLMs and
non-parametric information provided in the prompt context. In this work we ask
what are the desiderata for LLMs when a knowledge conflict arises and whether
existing LLMs fulfill them. We posit that LLMs should 1) identify knowledge
conflicts, 2) pinpoint conflicting information segments, and 3) provide
distinct answers or viewpoints in conflicting scenarios. To this end, we
introduce KNOWLEDGE CONFLICT, an evaluation framework for simulating contextual
knowledge conflicts and quantitatively evaluating to what extent LLMs achieve
these goals. KNOWLEDGE CONFLICT includes diverse and complex situations of
knowledge conflict, knowledge from diverse entities and domains, two synthetic
conflict creation methods, and settings with progressively increasing
difficulty to reflect realistic knowledge conflicts. Extensive experiments with
the KNOWLEDGE CONFLICT framework reveal that while LLMs perform well in
identifying the existence of knowledge conflicts, they struggle to determine
the specific conflicting knowledge and produce a response with distinct answers
amidst conflicting information. To address these challenges, we propose new
instruction-based approaches that augment LLMs to better achieve the three
goals. Further analysis shows that abilities to tackle knowledge conflicts are
greatly impacted by factors such as knowledge domain and prompt text, while
generating robust responses to knowledge conflict scenarios remains an open
research question.
Related papers
- From Internal Conflict to Contextual Adaptation of Language Models [42.776896363518844]
Studies indicate that LMs often ignore the provided context as it can conflict with the pre-existing LM's memory learned during pre-training.
We introduce the DYNAMICQA dataset, which includes facts with a temporal dynamic nature.
Our experiments reveal that static facts, which are unlikely to change, are more easily updated with additional context.
arXiv Detail & Related papers (2024-07-24T06:06:07Z) - Untangle the KNOT: Interweaving Conflicting Knowledge and Reasoning Skills in Large Language Models [51.72963030032491]
Knowledge documents for large language models (LLMs) may conflict with the memory of LLMs due to outdated or incorrect knowledge.
We construct a new dataset, dubbed KNOT, for knowledge conflict resolution examination in the form of question answering.
arXiv Detail & Related papers (2024-04-04T16:40:11Z) - Knowledge Conflicts for LLMs: A Survey [24.731074825915833]
Survey focuses on three categories of knowledge conflicts: context-memory, inter-context, and intra-memory conflict.
These conflicts can significantly impact the trustworthiness and performance of large language models.
arXiv Detail & Related papers (2024-03-13T08:02:23Z) - Don't Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaboration [39.603649838876294]
We study approaches to identify LLM knowledge gaps and abstain from answering questions when knowledge gaps are present.
Motivated by their failures in self-reflection and over-reliance on held-out sets, we propose two novel approaches.
arXiv Detail & Related papers (2024-02-01T06:11:49Z) - A Comprehensive Study of Knowledge Editing for Large Language Models [82.65729336401027]
Large Language Models (LLMs) have shown extraordinary capabilities in understanding and generating text that closely mirrors human communication.
This paper defines the knowledge editing problem and provides a comprehensive review of cutting-edge approaches.
We introduce a new benchmark, KnowEdit, for a comprehensive empirical evaluation of representative knowledge editing approaches.
arXiv Detail & Related papers (2024-01-02T16:54:58Z) - RECALL: A Benchmark for LLMs Robustness against External Counterfactual
Knowledge [69.79676144482792]
This study aims to evaluate the ability of LLMs to distinguish reliable information from external knowledge.
Our benchmark consists of two tasks, Question Answering and Text Generation, and for each task, we provide models with a context containing counterfactual information.
arXiv Detail & Related papers (2023-11-14T13:24:19Z) - Beyond Factuality: A Comprehensive Evaluation of Large Language Models
as Knowledge Generators [78.63553017938911]
Large language models (LLMs) outperform information retrieval techniques for downstream knowledge-intensive tasks.
However, community concerns abound regarding the factuality and potential implications of using this uncensored knowledge.
We introduce CONNER, designed to evaluate generated knowledge from six important perspectives.
arXiv Detail & Related papers (2023-10-11T08:22:37Z) - Knowledge Crosswords: Geometric Knowledge Reasoning with Large Language Models [49.23348672822087]
We propose Knowledge Crosswords, a benchmark consisting of incomplete knowledge networks bounded by structured factual constraints.
The novel setting of geometric knowledge reasoning necessitates new LM abilities beyond existing atomic/linear multi-hop QA.
We conduct extensive experiments to evaluate existing LLMs and approaches on Knowledge Crosswords.
arXiv Detail & Related papers (2023-10-02T15:43:53Z) - "Merge Conflicts!" Exploring the Impacts of External Distractors to
Parametric Knowledge Graphs [15.660128743249611]
Large language models (LLMs) acquire extensive knowledge during pre-training, known as their parametric knowledge.
LLMs inevitably require external knowledge during their interactions with users.
This raises a crucial question: How will LLMs respond when external knowledge interferes with their parametric knowledge?
arXiv Detail & Related papers (2023-09-15T17:47:59Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.