Edit Knowledge, Not Just Facts via Multi-Step Reasoning over Background Stories

• URL: http://arxiv.org/abs/2602.02028v1
• Date: Mon, 02 Feb 2026 12:22:51 GMT
• Title: Edit Knowledge, Not Just Facts via Multi-Step Reasoning over Background Stories
• Authors: Ya Gao, Kalle Kujanpää, Pekka Marttinen, Harri Valpola, Alexander Ilin,
• Abstract summary: We argue that knowledge internalization is a reasoning problem rather than a memorization problem.<n>We propose a training strategy based on three principles.<n> Experiments show that models trained with this strategy effectively leverage newly acquired knowledge during reasoning.
• Score: 54.75026632028614
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Enabling artificial intelligence systems, particularly large language models, to integrate new knowledge and flexibly apply it during reasoning remains a central challenge. Existing knowledge editing approaches emphasize atomic facts, improving factual recall but often failing to integrate new information into a coherent framework usable across contexts. In this work, we argue that knowledge internalization is fundamentally a reasoning problem rather than a memorization problem. Consequently, a model should be trained in situations where the new information is instrumental to solving a task, combined with pre-existing knowledge, and exercised through multi-step reasoning. Based on this insight, we propose a training strategy based on three principles. First, new knowledge is introduced as a coherent background story that contextualizes novel facts and explains their relation to existing knowledge. Second, models are trained using self-generated multi-hop questions that require multi-step reasoning involving the new information. Third, training is done using knowledge distillation, forcing a student model to internalize the teacher's reasoning behavior without access to the novel information. Experiments show that models trained with this strategy effectively leverage newly acquired knowledge during reasoning and achieve remarkable performance on challenging questions that require combining multiple new facts.

Related papers

• Stable Knowledge Editing in Large Language Models [68.98582618305679]
  We introduce StableKE, a knowledge editing method based on knowledge augmentation rather than knowledge localization. To overcome the expense of human labeling, StableKE integrates two automated knowledge augmentation strategies. StableKE surpasses other knowledge editing methods, demonstrating stability both edited knowledge and multi-hop knowledge.
  arXiv  Detail & Related papers  (2024-02-20T14:36:23Z)
• The Effect of Masking Strategies on Knowledge Retention by Language Models [9.130890741447422]
  This paper aims to understand the effect of pre-training tasks on the amount of knowledge captured and forgotten by language models. We test the model's knowledge retention by measuring its ability to answer factual questions. Our findings demonstrate that, like the ability to perform a task, the knowledge acquired from being trained on that task is forgotten when a model is trained to perform another task.
  arXiv  Detail & Related papers  (2023-06-12T15:35:23Z)
• Leveraging Skill-to-Skill Supervision for Knowledge Tracing [13.753990664747265]
  Knowledge tracing plays a pivotal role in intelligent tutoring systems. Recent advances in knowledge tracing models have enabled better exploitation of problem solving history. Knowledge tracing algorithms that incorporate knowledge directly are important to settings with limited data or cold starts.
  arXiv  Detail & Related papers  (2023-06-12T03:23:22Z)
• RECKONING: Reasoning through Dynamic Knowledge Encoding [51.076603338764706]
  We show that language models can answer questions by reasoning over knowledge provided as part of the context. In these situations, the model fails to distinguish the knowledge that is necessary to answer the question. We propose teaching the model to reason more robustly by folding the provided contextual knowledge into the model's parameters.
  arXiv  Detail & Related papers  (2023-05-10T17:54:51Z)
• Adaptively Integrated Knowledge Distillation and Prediction Uncertainty for Continual Learning [71.43841235954453]
  Current deep learning models often suffer from catastrophic forgetting of old knowledge when continually learning new knowledge. Existing strategies to alleviate this issue often fix the trade-off between keeping old knowledge (stability) and learning new knowledge (plasticity)
  arXiv  Detail & Related papers  (2023-01-18T05:36:06Z)
• Anti-Retroactive Interference for Lifelong Learning [65.50683752919089]
  We design a paradigm for lifelong learning based on meta-learning and associative mechanism of the brain. It tackles the problem from two aspects: extracting knowledge and memorizing knowledge. It is theoretically analyzed that the proposed learning paradigm can make the models of different tasks converge to the same optimum.
  arXiv  Detail & Related papers  (2022-08-27T09:27:36Z)
• Knowledge-Grounded Dialogue with Reward-Driven Knowledge Selection [1.1633929083694388]
  Knoformer is a dialogue response generation model based on reinforcement learning. It can automatically select one or more related knowledge from the knowledge pool and does not need knowledge labels during training.
  arXiv  Detail & Related papers  (2021-08-31T08:53:08Z)
• Incremental Knowledge Based Question Answering [52.041815783025186]
  We propose a new incremental KBQA learning framework that can progressively expand learning capacity as humans do. Specifically, it comprises a margin-distilled loss and a collaborative selection method, to overcome the catastrophic forgetting problem. The comprehensive experiments demonstrate its effectiveness and efficiency when working with the evolving knowledge base.
  arXiv  Detail & Related papers  (2021-01-18T09:03:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.