Reason-KE++: Aligning the Process, Not Just the Outcome, for Faithful LLM Knowledge Editing

• URL: http://arxiv.org/abs/2511.12661v1
• Date: Sun, 16 Nov 2025 15:49:01 GMT
• Title: Reason-KE++: Aligning the Process, Not Just the Outcome, for Faithful LLM Knowledge Editing
• Authors: Yuchen Wu, Liang Ding, Li Shen, Dacheng Tao,
• Abstract summary: We find that SFT-based methods, e.g., Reason-KE, suffer from a "faithfulness gap"<n>This gap enables the LLM's powerful parametric priors to override new contextual facts.<n>We propose Reason-KE++, an SFT+RL framework that instills process-level faithfulness.
• Score: 63.96040994220329
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Aligning Large Language Models (LLMs) to be faithful to new knowledge in complex, multi-hop reasoning tasks is a critical, yet unsolved, challenge. We find that SFT-based methods, e.g., Reason-KE, while state-of-the-art, suffer from a "faithfulness gap": they optimize for format mimicry rather than sound reasoning. This gap enables the LLM's powerful parametric priors to override new contextual facts, resulting in critical factual hallucinations (e.g., incorrectly reasoning "Houston" from "NASA" despite an explicit edit). To solve this core LLM alignment problem, we propose Reason-KE++, an SFT+RL framework that instills process-level faithfulness. Its core is a Stage-aware Reward mechanism that provides dense supervision for intermediate reasoning steps (e.g., Decomposition, Sub-answer Correctness). Crucially, we identify that naive outcome-only RL is a deceptive trap for LLM alignment: it collapses reasoning integrity (e.g., 19.00% Hop acc) while superficially boosting final accuracy. Our process-aware framework sets a new SOTA of 95.48% on MQUAKE-CF-3k (+5.28%), demonstrating that for complex tasks, aligning the reasoning process is essential for building trustworthy LLMs.

Related papers

• Towards Generalizable Reasoning: Group Causal Counterfactual Policy Optimization for LLM Reasoning [50.352417879912515]
  Large language models (LLMs) excel at complex tasks with advances in reasoning capabilities.<n>We propose Group Causal Counterfactual Policy Optimization to explicitly train LLMs to learn generalizable reasoning patterns.<n>We then construct token-level advantages from this reward and optimize the policy, encouraging LLMs to favor reasoning patterns that are process-valid and counterfactually robust.
  arXiv  Detail & Related papers  (2026-02-06T08:03:11Z)
• Can Aha Moments Be Fake? Identifying True and Decorative Thinking Steps in Chain-of-Thought [72.45900226435289]
  Large language models (LLMs) can generate long Chain-of-Thought (CoT) at test time, enabling them to solve complex tasks.<n>We measure the step-wise causal influence of each reasoning step on the model's final prediction with a proposed True Thinking Score (TTS)<n>We identify a TrueThinking direction in the latent space of LLMs, which can force the model to perform or disregard certain CoT steps.
  arXiv  Detail & Related papers  (2025-10-28T20:14:02Z)
• WakenLLM: Evaluating Reasoning Potential and Stability in LLMs via Fine-Grained Benchmarking [34.350505059394536]
  Large Language Models (LLMs) frequently output the label Unknown in reasoning tasks.<n>We introduce WakenLLM, a framework that quantifies the portion of Unknown output attributable to model incapacity.
  arXiv  Detail & Related papers  (2025-07-22T03:21:48Z)
• Misaligning Reasoning with Answers -- A Framework for Assessing LLM CoT Robustness [3.9930400744726273]
  We design a novel evaluation framework, MATCHA, to investigate the relationship between answer and reasoning.<n>In domains like education and healthcare, reasoning is key for model trustworthiness.<n>Our results show that LLMs exhibit greater vulnerability to input perturbations for multi-step and commonsense tasks than compared to logical tasks.
  arXiv  Detail & Related papers  (2025-05-23T02:42:16Z)
• Have Large Language Models Learned to Reason? A Characterization via 3-SAT Phase Transition [11.422434149376478]
  Large Language Models (LLMs) have been touted as AI models possessing advanced reasoning abilities.<n>In theory, autoregressive LLMs with Chain-of-Thought (CoT) can perform more serial computations to solve complex reasoning tasks.<n>Recent studies suggest that, despite this capacity, LLMs do not truly learn to reason but instead fit on statistical features.
  arXiv  Detail & Related papers  (2025-04-04T20:57:36Z)
• SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMs [48.28847964704554]
  Chain-of-Thought (CoT) reasoning enables Large Language Models (LLMs) to solve complex reasoning tasks.<n>We propose a novel approach for continuous-space reasoning that does not require modifying the LLM.
  arXiv  Detail & Related papers  (2025-02-17T18:52:29Z)
• Automatic Curriculum Expert Iteration for Reliable LLM Reasoning [60.60318625779015]
  Hallucinations (i.e., generating plausible but inaccurate content) and laziness (i.e. excessive refusals or defaulting to "I don't know") persist as major challenges in LLM reasoning.<n>Current efforts to reduce hallucinations primarily focus on factual errors in knowledge-grounded tasks, often neglecting hallucinations related to faulty reasoning.<n>We propose Automatic Curriculum Expert Iteration (Auto-CEI) to enhance LLM reasoning and align responses to the model's capabilities.
  arXiv  Detail & Related papers  (2024-10-10T05:43:07Z)
• Can Large Language Models Reason? A Characterization via 3-SAT [11.422434149376478]
  Large Language Models (LLMs) have been touted as AI models possessing advanced reasoning abilities. Recent works have shown that LLMs often bypass true reasoning using shortcuts, sparking skepticism. We propose an experimental protocol centered on 3-SAT -- the NP-complete problem lying at the core of logical reasoning and constraint satisfaction tasks.
  arXiv  Detail & Related papers  (2024-08-13T21:54:10Z)
• Neuro-Symbolic Integration Brings Causal and Reliable Reasoning Proofs [95.07757789781213]
  Two lines of approaches are adopted for complex reasoning with LLMs.<n>One line of work prompts LLMs with various reasoning structures, while the structural outputs can be naturally regarded as intermediate reasoning steps.<n>The other line of work adopt LLM-free declarative solvers to do the reasoning task, rendering higher reasoning accuracy but lacking interpretability due to the black-box nature of the solvers.<n>We present a simple extension to the latter line of work. Specifically, we showcase that the intermediate search logs generated by Prolog interpreters can be accessed and interpreted into human-readable reasoning.
  arXiv  Detail & Related papers  (2023-11-16T11:26:21Z)
• Language Models with Rationality [57.37201135072838]
  Large language models (LLMs) are proficient at question-answering (QA) It is not always clear how (or even if) an answer follows from their latent "beliefs"
  arXiv  Detail & Related papers  (2023-05-23T17:04:25Z)
• Can ChatGPT Defend its Belief in Truth? Evaluating LLM Reasoning via Debate [19.887103433032774]
  Large language models (LLMs) have shown impressive performance in complex reasoning tasks. This work explores testing LLMs' reasoning by engaging with them in a debate-like conversation. We find that despite their impressive performance, LLMs like ChatGPT cannot maintain their beliefs in truth for a significant portion of examples.
  arXiv  Detail & Related papers  (2023-05-22T15:47:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.