From Internal Conflict to Contextual Adaptation of Language Models

• URL: http://arxiv.org/abs/2407.17023v1
• Date: Wed, 24 Jul 2024 06:06:07 GMT
• Title: From Internal Conflict to Contextual Adaptation of Language Models
• Authors: Sara Vera Marjanović, Haeun Yu, Pepa Atanasova, Maria Maistro, Christina Lioma, Isabelle Augenstein,
• Abstract summary: Studies indicate that LMs often ignore the provided context as it can conflict with the pre-existing LM's memory learned during pre-training. We introduce the DYNAMICQA dataset, which includes facts with a temporal dynamic nature. Our experiments reveal that static facts, which are unlikely to change, are more easily updated with additional context.
• Score: 42.776896363518844
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Knowledge-intensive language understanding tasks require Language Models (LMs) to integrate relevant context, mitigating their inherent weaknesses, such as incomplete or outdated knowledge. Nevertheless, studies indicate that LMs often ignore the provided context as it can conflict with the pre-existing LM's memory learned during pre-training. Moreover, conflicting knowledge can already be present in the LM's parameters, termed intra-memory conflict. Existing works have studied the two types of knowledge conflicts only in isolation. We conjecture that the (degree of) intra-memory conflicts can in turn affect LM's handling of context-memory conflicts. To study this, we introduce the DYNAMICQA dataset, which includes facts with a temporal dynamic nature where a fact can change with a varying time frequency and disputable dynamic facts, which can change depending on the viewpoint. DYNAMICQA is the first to include real-world knowledge conflicts and provide context to study the link between the different types of knowledge conflicts. With the proposed dataset, we assess the use of uncertainty for measuring the intra-memory conflict and introduce a novel Coherent Persuasion (CP) score to evaluate the context's ability to sway LM's semantic output. Our extensive experiments reveal that static facts, which are unlikely to change, are more easily updated with additional context, relative to temporal and disputable facts.

Related papers

• What Is Seen Cannot Be Unseen: The Disruptive Effect of Knowledge Conflict on Large Language Models [16.41477610681199]
  Large language models frequently rely on both contextual input and parametric knowledge to perform tasks.<n>These sources can come into conflict, especially when retrieved documents contradict the model's parametric beliefs.<n>We propose a diagnostic framework to systematically evaluate LLM behavior under context-memory conflict.
  arXiv  Detail & Related papers  (2025-06-06T19:20:23Z)
• Conflicts in Texts: Data, Implications and Challenges [58.03478157713084]
  Conflicts could reflect the complexity of situations, changes that need to be explained and dealt with, difficulties in data annotation, and mistakes in generated outputs. This survey categorizes these conflicts into three key areas: (1) natural texts on the web, where factual inconsistencies, subjective biases, and multiple perspectives introduce contradictions; (2) human-annotated data, where annotator disagreements, mistakes, and societal biases impact model training; and (3) model interactions, where hallucinations and knowledge conflicts emerge during deployment. We highlight key challenges and future directions for developing conflict-aware NLP systems that can reason over and reconcile conflicting information more effectively
  arXiv  Detail & Related papers  (2025-04-28T04:24:01Z)
• Mitigating Knowledge Conflicts in Language Model-Driven Question Answering [15.29366851382021]
  Two fundamental knowledge sources play crucial roles in document-based question answering and document summarization systems. Recent studies revealed a significant challenge: when there exists a misalignment between the model's inherent knowledge and the ground truth answers in training data, the system may exhibit problematic behaviors during inference. Our investigation proposes a strategy to minimize hallucination by building explicit connection between source inputs and generated outputs.
  arXiv  Detail & Related papers  (2024-11-18T07:33:10Z)
• Analysing the Residual Stream of Language Models Under Knowledge Conflicts [23.96385393039587]
  Large language models (LLMs) can store a significant amount of factual knowledge in their parameters. However, their parametric knowledge may conflict with the information provided in the context. This can lead to undesirable model behaviour, such as reliance on outdated or incorrect information.
  arXiv  Detail & Related papers  (2024-10-21T15:12:51Z)
• ECon: On the Detection and Resolution of Evidence Conflicts [56.89209046429291]
  The rise of large language models (LLMs) has significantly influenced the quality of information in decision-making systems. This study introduces a method for generating diverse, validated evidence conflicts to simulate real-world misinformation scenarios.
  arXiv  Detail & Related papers  (2024-10-05T07:41:17Z)
• LLMs' Reading Comprehension Is Affected by Parametric Knowledge and Struggles with Hypothetical Statements [59.71218039095155]
  Task of reading comprehension (RC) provides a primary means to assess language models' natural language understanding (NLU) capabilities. If the context aligns with the models' internal knowledge, it is hard to discern whether the models' answers stem from context comprehension or from internal information. To address this issue, we suggest to use RC on imaginary data, based on fictitious facts and entities.
  arXiv  Detail & Related papers  (2024-04-09T13:08:56Z)
• Cutting Off the Head Ends the Conflict: A Mechanism for Interpreting and Mitigating Knowledge Conflicts in Language Models [18.2500350157507]
  Internal memory and external context inevitably clash, leading to knowledge conflicts within language models (LMs) We propose a novel method called PatH PatcHing (PH3), which can efficiently mitigate knowledge conflicts by pruning conflicting attention heads without updating model parameters.
  arXiv  Detail & Related papers  (2024-02-28T08:34:41Z)
• Discerning and Resolving Knowledge Conflicts through Adaptive Decoding with Contextual Information-Entropy Constraint [20.543282448771336]
  We propose an adaptive decoding method to discern whether the knowledge conflicts occur and resolve them. Experiments show that COIECD exhibits strong performance and robustness over knowledge conflicts in realistic datasets.
  arXiv  Detail & Related papers  (2024-02-19T07:10:30Z)
• A Glitch in the Matrix? Locating and Detecting Language Model Grounding with Fakepedia [57.31074448586854]
  Large language models (LLMs) have an impressive ability to draw on novel information supplied in their context. Yet the mechanisms underlying this contextual grounding remain unknown. We present a novel method to study grounding abilities using Fakepedia.
  arXiv  Detail & Related papers  (2023-12-04T17:35:42Z)
• Resolving Knowledge Conflicts in Large Language Models [46.903549751371415]
  Large language models (LLMs) often encounter knowledge conflicts. We ask what are the desiderata for LLMs when a knowledge conflict arises and whether existing LLMs fulfill them. We introduce an evaluation framework for simulating contextual knowledge conflicts.
  arXiv  Detail & Related papers  (2023-10-02T06:57:45Z)
• Getting Sick After Seeing a Doctor? Diagnosing and Mitigating Knowledge Conflicts in Event Temporal Reasoning [87.92209048521153]
  Event temporal reasoning aims at identifying the temporal relations between two or more events from narratives. Knowledge conflicts arise when there is a mismatch between the actual temporal relations of events in the context and the prior knowledge or biases learned by the model.
  arXiv  Detail & Related papers  (2023-05-24T10:04:06Z)
• Large Language Models with Controllable Working Memory [64.71038763708161]
  Large language models (LLMs) have led to a series of breakthroughs in natural language processing (NLP) What further sets these models apart is the massive amounts of world knowledge they internalize during pretraining. How the model's world knowledge interacts with the factual information presented in the context remains under explored.
  arXiv  Detail & Related papers  (2022-11-09T18:58:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.