On the Hidden Costs of Counterfactual Knowledge Training in LLM Unlearning

• URL: http://arxiv.org/abs/2605.27083v1
• Date: Tue, 26 May 2026 14:34:14 GMT
• Title: On the Hidden Costs of Counterfactual Knowledge Training in LLM Unlearning
• Abstract summary: Counterfactual tuning (CFT) has emerged as a promising paradigm for Large Language Model (LLM) unlearning by training models.<n>We find that this paradigm still underperforms other paradigms in some aspects.<n>We introduce RWKU+, an extended benchmark equipped with novel trade-off metrics and gradient-level diagnostic tools.
• Score: 65.58724910851039
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Counterfactual tuning (CFT) has emerged as a promising paradigm for Large Language Model (LLM) unlearning by training models to generate alternative fictitious knowledge in place of undesired content. However, in this work, we find that this paradigm still underperforms other paradigms in some aspects, and identify two previously overlooked pitfalls underlying this gap: (1) knowledge conflict, where mutual inconsistencies within counterfactual corpora induce conflicting gradients that disrupt parameter optimization, and (2) hallucination spillover, where fitting false targets instills a persistent fabrication bias, inflating hallucination rates on unrelated domains. To systematically diagnose these issues, we introduce RWKU+, an extended benchmark equipped with novel trade-off metrics and gradient-level diagnostic tools. Our work further discusses the limitations and overhead of the paradigm, aiming to provide insights and actionable guidance for more rigorous LLM unlearning research.

Related papers

• Revitalizing the Beginning: Avoiding Storage Dependency for Model Merging in Continual Learning [57.10440766103372]
  Trajectory Regularized Merging (TRM) is a framework that reformulates the merging phase as an optimization process within an augmented trajectory subspace.<n>Our framework integrates three synergistic objectives including task alignment, prediction consistency, and gradient responsiveness to concurrently preserve merged model's historical stability and re-activate optimization dynamics.
  arXiv  Detail & Related papers  (2026-05-08T14:07:32Z)
• Diversity in Large Language Models under Supervised Fine-Tuning [15.119045051735633]
  Supervised Fine-Tuning (SFT) is essential for aligning Large Language Models (LLMs) with user intent, yet it is believed to suppress generative diversity.<n>In this study, we attribute the decline to two primary drivers: the forgetting of preexisting knowledge and low-frequency patterns within finetuning datasets.<n>Our evaluation confirms at scale that generation narrows after SFT and strengthens the hypothesis explaining this effect.
  arXiv  Detail & Related papers  (2026-04-30T20:20:59Z)
• Harmonizing Multi-Objective LLM Unlearning via Unified Domain Representation and Bidirectional Logit Distillation [11.723777730116831]
  Large Language Models (LLMs) unlearning is crucial for removing hazardous or privacy-leaking information from the model.<n>Existing unlearning methods primarily focus on a limited subset of these goals.<n>We propose a novel multi-objective unlearning framework that harmonizes multiple unlearning objectives.
  arXiv  Detail & Related papers  (2026-04-16T19:09:17Z)
• DIS2: Disentanglement Meets Distillation with Classwise Attention for Robust Remote Sensing Segmentation under Missing Modalities [28.992992584085787]
  DIS2 is a new paradigm shifting from modality-shared feature dependence to active, guided missing features compensation.<n> Compensatory features are explicitly captured which, when fused with the features of the available modality, approximate the ideal fused representation of the full-modality case.<n>Our proposed approach significantly outperforms state-of-the-art methods across benchmarks.
  arXiv  Detail & Related papers  (2026-01-20T01:33:54Z)
• Ground What You See: Hallucination-Resistant MLLMs via Caption Feedback, Diversity-Aware Sampling, and Conflict Regularization [38.469173375694076]
  This paper systematically analyzes the root causes of hallucinations in Multimodal Large Language Models (MLLMs)<n>It identifies three critical factors: (1) an over-reliance on chained visual reasoning, where inaccurate initial descriptions anchor subsequent inferences to incorrect premises; (2) insufficient exploration diversity during policy optimization, leading the model to generate overly confident but erroneous outputs; and (3) destructive conflicts between training samples, where NTK similarity causes false associations and unstable parameter updates.<n> Experimental results demonstrate that our proposed method significantly reduces hallucination rates and effectively enhances the inference accuracy of MLLMs.
  arXiv  Detail & Related papers  (2026-01-09T07:59:18Z)
• Understanding Catastrophic Interference: On the Identifibility of Latent Representations [67.05452287233122]
  Catastrophic interference, also known as catastrophic forgetting, is a fundamental challenge in machine learning.<n>We propose a novel theoretical framework that formulates catastrophic interference as an identification problem.<n>Our approach provides both theoretical guarantees and practical performance improvements across both synthetic and benchmark datasets.
  arXiv  Detail & Related papers  (2025-09-27T00:53:32Z)
• Injecting Imbalance Sensitivity for Multi-Task Learning [36.60453299563175]
  Multi-task learning (MTL) has emerged as a promising approach for deploying deep learning models in real-life applications.<n>Recent studies have proposed optimization-based learning paradigms to establish task-shared representations in MTL.<n>Our paper empirically argues that these studies primarily emphasize the conflict issue while neglecting the potentially more significant impact of imbalance/dominance in MTL.
  arXiv  Detail & Related papers  (2025-03-11T03:11:54Z)
• A Two-Stage Learning-to-Defer Approach for Multi-Task Learning [3.4289478404209826]
  We introduce a novel Two-Stage L2D framework for multi-task learning that integrates classification and regression through a unified deferral mechanism.<n>Our method leverages a two-stage surrogate loss family, which we prove to be both Bayes-consistent and $(mathcalG, mathcalR)$-consistent.
  arXiv  Detail & Related papers  (2024-10-21T07:44:57Z)
• Temporal-Difference Variational Continual Learning [77.92320830700797]
  We propose new learning objectives that integrate the regularization effects of multiple previous posterior estimations.<n>Our approach effectively mitigates Catastrophic Forgetting, outperforming strong Variational CL methods.
  arXiv  Detail & Related papers  (2024-10-10T10:58:41Z)
• MR-Ben: A Meta-Reasoning Benchmark for Evaluating System-2 Thinking in LLMs [55.20845457594977]
  Large language models (LLMs) have shown increasing capability in problem-solving and decision-making.<n>We present a process-based benchmark MR-Ben that demands a meta-reasoning skill.<n>Our meta-reasoning paradigm is especially suited for system-2 slow thinking.
  arXiv  Detail & Related papers  (2024-06-20T03:50:23Z)
• Towards Effective Evaluations and Comparisons for LLM Unlearning Methods [97.2995389188179]
  This paper seeks to refine the evaluation of machine unlearning for large language models.<n>It addresses two key challenges -- the robustness of evaluation metrics and the trade-offs between competing goals.
  arXiv  Detail & Related papers  (2024-06-13T14:41:00Z)
• MR-GSM8K: A Meta-Reasoning Benchmark for Large Language Model Evaluation [60.65820977963331]
  We introduce a novel evaluation paradigm for Large Language Models (LLMs) This paradigm shifts the emphasis from result-oriented assessments, which often neglect the reasoning process, to a more comprehensive evaluation. By applying this paradigm in the GSM8K dataset, we have developed the MR-GSM8K benchmark.
  arXiv  Detail & Related papers  (2023-12-28T15:49:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.