Popular Knowledge Propagates More Errors in LLM Knowledge Updating

• URL: http://arxiv.org/abs/2609.08067v1
• Date: Tue, 08 Sep 2026 00:23:27 GMT
• Title: Popular Knowledge Propagates More Errors in LLM Knowledge Updating
• Abstract summary: Updating a language model's knowledge through fine-tuning can induce factual forgetting and new hallucinations.<n>We investigate which facts are most vulnerable to collateral corruption during other updates.<n>We propose Popularity-based Anchoring (PopAnchor), a lightweight rehearsal strategy that preserves a small set of popular facts and reduces forgetting.
• Score: 64.62006223921576
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Updating a language model's knowledge through fine-tuning is essential for keeping its outputs current, yet can also induce factual forgetting and new hallucinations. Prior work shows that long-tail knowledge is harder to acquire and newly memorized long-tail facts are difficult to retain during later fine-tuning. We study a complementary question: among facts that a model has encoded correctly, which are most vulnerable to collateral corruption during other updates? To investigate this question under a realistic factual distribution, we construct a large-scale graph FACTPROP of verified Wikipedia facts by linking triples that share head or tail entities, thereby preserving connections among factual knowledge. We fine-tune models on factual statements and measure correct-to-incorrect facts after each update. Our results reveal a pattern distinct from prior findings on long-tail vulnerability during acquisition and retention: among facts that models already answer correctly, those associated with highly connected entities are more likely to be corrupted by neighboring updates, and updates to such facts propagate errors more broadly. Structural popularity therefore predicts both vulnerability and downstream damage. Inspired by this finding, we propose Popularity-based Anchoring (PopAnchor), a lightweight rehearsal strategy that preserves a small set of popular facts and reduces forgetting.

Related papers

• NewsEdits 2.0: Learning the Intentions Behind Updating News [74.84017890548259]
  As events progress, news articles often update with new information: if we are not cautious, we risk propagating outdated facts.<n>In this work, we hypothesize that linguistic features indicate factual fluidity, and that we can predict which facts in a news article will update using solely the text of a news article.
  arXiv  Detail & Related papers  (2024-11-27T23:35:23Z)
• Learning and Unlearning of Fabricated Knowledge in Language Models [16.971082623826263]
  We show that facts that conflict with common knowledge are remembered for tens of thousands of training steps. We show that impacts of knowledge-conflicting facts in LMs, though they can be long lasting, can be largely erased by novel application of multi-step sparse updates.
  arXiv  Detail & Related papers  (2024-10-29T05:33:14Z)
• Understanding Finetuning for Factual Knowledge Extraction [45.398982602347765]
  We show that fine-tuning on lesser-known facts that are poorly stored during pretraining yields significantly worse factuality than fine-tuning on well-known facts, even when all facts are seen during pretraining. Our results shed light on the interaction between pretrained knowledge and finetuning data and demonstrate the importance of taking into account how facts are stored in the pretrained model when fine-tuning for knowledge-intensive tasks.
  arXiv  Detail & Related papers  (2024-06-20T23:27:06Z)
• Outdated Issue Aware Decoding for Reasoning Questions on Edited Knowledge [93.54427119091174]
  We propose outDated ISsue aware deCOding to enhance the performance of edited models on reasoning questions. We capture the difference in the probability distribution between the original and edited models. We amplify the difference of the token prediction in the edited model to alleviate the outdated issue.
  arXiv  Detail & Related papers  (2024-06-05T03:00:15Z)
• Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations? [33.702498916775426]
  We study the impact of new knowledge on the capability of the fine-tuned model to utilize its pre-existing knowledge. We demonstrate that large language models struggle to acquire new factual knowledge through fine-tuning. As the examples with new knowledge are eventually learned, they linearly increase the model's tendency to hallucinate.
  arXiv  Detail & Related papers  (2024-05-09T17:00:22Z)
• Know When To Stop: A Study of Semantic Drift in Text Generation [9.76171773410722]
  We show that modern LLMs tend to generate correct facts first, then "drift away" and generate incorrect facts later. This correct-then-incorrect generation pattern suggests that factual accuracy can be improved by knowing when to stop generation.
  arXiv  Detail & Related papers  (2024-04-08T11:25:30Z)
• MuLan: A Study of Fact Mutability in Language Models [50.626787909759976]
  Trustworthy language models ideally identify mutable facts as such and process them accordingly. We create MuLan, a benchmark for evaluating the ability of English language models to anticipate time-contingency.
  arXiv  Detail & Related papers  (2024-04-03T19:47:33Z)
• Mitigating Temporal Misalignment by Discarding Outdated Facts [58.620269228776294]
  Large language models are often used under temporal misalignment, tasked with answering questions about the present. We propose fact duration prediction: the task of predicting how long a given fact will remain true. Our data and code are released publicly at https://github.com/mikejqzhang/mitigating_misalignment.
  arXiv  Detail & Related papers  (2023-05-24T07:30:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.