That's Deprecated! Understanding, Detecting, and Steering Knowledge Conflicts in Language Models for Code Generation

• URL: http://arxiv.org/abs/2510.19116v1
• Date: Tue, 21 Oct 2025 22:27:56 GMT
• Title: That's Deprecated! Understanding, Detecting, and Steering Knowledge Conflicts in Language Models for Code Generation
• Authors: Jaesung Bae, Cameron Churchwell, Mitchell Hermon, Tsun-An Hsieh, Jocelyn Xu, Yekaterina Yegorova, Mark Hasegawa-Johnson, Heng Ji,
• Abstract summary: Large language models (LLMs) behave when faced with discrepancies between their parametric knowledge and conflicting information contained in a prompt.<n>We propose a domain-agnostic framework for constructing and interpreting such conflicts.<n>We show that activation-level steering can achieve up to a textbf12.6% improvement in steering success over a random baseline.
• Score: 55.78914774437411
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper investigates how large language models (LLMs) behave when faced with discrepancies between their parametric knowledge and conflicting information contained in a prompt. Building on prior question-answering (QA) research, we extend the investigation of knowledge conflicts to the realm of code generation. We propose a domain-agnostic framework for constructing and interpreting such conflicts, along with a novel evaluation method and dataset tailored to code conflict scenarios. Our experiments indicate that sufficiently large LLMs encode the notion of a knowledge conflict in their parameters, enabling us to detect knowledge conflicts with up to \textbf{80.65\%} accuracy. Building on these insights, we show that activation-level steering can achieve up to a \textbf{12.6\%} improvement in steering success over a random baseline. However, effectiveness depends critically on balancing model size, task domain, and steering direction. The experiment code and data will be made publicly available after acceptance.

Related papers

• CC-VQA: Conflict- and Correlation-Aware Method for Mitigating Knowledge Conflict in Knowledge-Based Visual Question Answering [53.7094431951084]
  Knowledge-based visual question answering (KB-VQA) demonstrates significant potential for handling knowledge-intensive tasks.<n>Conflicts arise between static parametric knowledge in vision language models and dynamically retrieved information.<n>We propose textbfCC-VQA as a training-free, conflict- and correlation-aware method for KB-VQA.
  arXiv  Detail & Related papers  (2026-02-27T11:56:26Z)
• FaithfulRAG: Fact-Level Conflict Modeling for Context-Faithful Retrieval-Augmented Generation [37.28571879699906]
  Large language models (LLMs) augmented with retrieval systems have demonstrated significant potential in handling knowledge-intensive tasks.<n>This paper proposes FaithfulRAG, a novel framework that resolves knowledge conflicts by explicitly modeling discrepancies between the models parametric knowledge and retrieved context.
  arXiv  Detail & Related papers  (2025-06-10T16:02:54Z)
• Task Matters: Knowledge Requirements Shape LLM Responses to Context-Memory Conflict [13.091464232666835]
  Large Language Models require both contextual knowledge and parametric memory, but these sources can disagree.<n>We study this question with a model-agnostic diagnostic framework that automatically detects disagreements between a model's beliefs and a curated knowledge set.<n>We find that performance degradation from conflict correlates with a task's knowledge reliance.
  arXiv  Detail & Related papers  (2025-06-06T19:20:23Z)
• Analysing the Residual Stream of Language Models Under Knowledge Conflicts [23.96385393039587]
  Large language models (LLMs) can store a significant amount of factual knowledge in their parameters.<n>However, their parametric knowledge may conflict with the information provided in the context.<n>This can lead to undesirable model behaviour, such as reliance on outdated or incorrect information.
  arXiv  Detail & Related papers  (2024-10-21T15:12:51Z)
• Unraveling Cross-Modality Knowledge Conflicts in Large Vision-Language Models [33.76903352835436]
  Large Vision-Language Models (LVLMs) have demonstrated impressive capabilities for capturing and reasoning over multimodal inputs. These models are prone to parametric knowledge conflicts, which arise from inconsistencies of represented knowledge between their vision and language components. We present a systematic approach to detect, interpret, and mitigate them.
  arXiv  Detail & Related papers  (2024-10-04T17:59:28Z)
• AdaCAD: Adaptively Decoding to Balance Conflicts between Contextual and Parametric Knowledge [57.66282463340297]
  Knowledge conflict arises from discrepancies between information in the context of a large language model and the knowledge stored in its parameters.<n>We propose a fine-grained, instance-level approach called AdaCAD, which dynamically infers the weight of adjustment based on the degree of conflict.<n>We show that ADACAD consistently outperforms other decoding baselines with average QA accuracy gains of 14.21% (absolute) over a static contrastive baseline, and improves the factuality of summaries by 6.19 (AlignScore)
  arXiv  Detail & Related papers  (2024-09-11T16:35:18Z)
• Studying Large Language Model Behaviors Under Context-Memory Conflicts With Real Documents [54.953320616069654]
  Retrieval-augmented generation mitigates many problems of fully parametric language models. In RAG, the model's knowledge can be updated from documents provided in context. We present a framework for studying such knowledge conflicts in a realistic setup.
  arXiv  Detail & Related papers  (2024-04-24T17:59:36Z)
• Robust and Scalable Model Editing for Large Language Models [75.95623066605259]
  We propose EREN (Edit models by REading Notes) to improve the scalability and robustness of LLM editing. Unlike existing techniques, it can integrate knowledge from multiple edits, and correctly respond to syntactically similar but semantically unrelated inputs.
  arXiv  Detail & Related papers  (2024-03-26T06:57:23Z)
• Discerning and Resolving Knowledge Conflicts through Adaptive Decoding with Contextual Information-Entropy Constraint [20.543282448771336]
  We propose an adaptive decoding method to discern whether the knowledge conflicts occur and resolve them. Experiments show that COIECD exhibits strong performance and robustness over knowledge conflicts in realistic datasets.
  arXiv  Detail & Related papers  (2024-02-19T07:10:30Z)
• Merging Generated and Retrieved Knowledge for Open-Domain QA [72.42262579925911]
  COMBO is a compatibility-Oriented knowledge Merging for Better Open-domain QA framework. We show that COMBO outperforms competitive baselines on three out of four tested open-domain QA benchmarks.
  arXiv  Detail & Related papers  (2023-10-22T19:37:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.