GONE: Structural Knowledge Unlearning via Neighborhood-Expanded Distribution Shaping

• URL: http://arxiv.org/abs/2603.12275v1
• Date: Sat, 21 Feb 2026 04:08:37 GMT
• Title: GONE: Structural Knowledge Unlearning via Neighborhood-Expanded Distribution Shaping
• Abstract summary: This paper introduces Graph Oblivion and Node Erasure (GONE), a benchmark for evaluating knowledge unlearning over structured knowledge graph (KG) facts in Large Language Models (LLMs)<n>This KG-based benchmark enables the disentanglement of three effects of unlearning: direct fact removal, reasoning-based leakage, and catastrophic forgetting.<n>NEDS, a novel unlearning framework, is designed to leverage graph connectivity and identify anchor correlated neighbors, enforcing a precise decision boundary between the forgotten fact and its semantic neighborhood.
• Score: 4.740962650068887
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Unlearning knowledge is a pressing and challenging task in Large Language Models (LLMs) because of their unprecedented capability to memorize and digest training data at scale, raising more significant issues regarding safety, privacy, and intellectual property. However, existing works, including parameter editing, fine-tuning, and distillation-based methods, are all focused on flat sentence-level data but overlook the relational, multi-hop, and reasoned knowledge in naturally structured data. In response to this gap, this paper introduces Graph Oblivion and Node Erasure (GONE), a benchmark for evaluating knowledge unlearning over structured knowledge graph (KG) facts in LLMs. This KG-based benchmark enables the disentanglement of three effects of unlearning: direct fact removal, reasoning-based leakage, and catastrophic forgetting. In addition, Neighborhood-Expanded Distribution Shaping (NEDS), a novel unlearning framework, is designed to leverage graph connectivity and identify anchor correlated neighbors, enforcing a precise decision boundary between the forgotten fact and its semantic neighborhood. Evaluations on LLaMA-3-8B and Mistral-7B across multiple knowledge editing and unlearning methods showcase NEDS's superior performance (1.000 on unlearning efficacy and 0.839 on locality) on GONE and other benchmarks. Code is available at https://anonymous.4open.science/r/GONE-4679/.

Related papers

• Detecting Hallucinations for Large Language Model-based Knowledge Graph Reasoning [48.237729567730696]
  We propose LUCID, the first halLUcination deteCtIon method for LLM-based knowleDge graph reasoning frameworks.<n> LUCID jointly leverages LLM attention scores, KG semantics, and structural information.<n> Experiments on nine datasets show that LUCID achieves state of the art performance compared to 15 baselines.
  arXiv  Detail & Related papers  (2026-04-27T12:20:28Z)
• GRADE: Probing Knowledge Gaps in LLMs through Gradient Subspace Dynamics [51.647572539626715]
  We present GRADE (Gradient Dynamics for knowledge gap detection), which quantifies the knowledge gap via the cross-layer rank ratio of the gradient to that of the corresponding hidden state subspace.<n>We validate modelname on six benchmarks, demonstrating its effectiveness and robustness to input perturbations.<n>In addition, we present a case study showing how the gradient chain can generate interpretable explanations of knowledge gaps for long-form answers.
  arXiv  Detail & Related papers  (2026-04-03T07:52:26Z)
• Knowledge Homophily in Large Language Models [75.12297135039776]
  We investigate an analogous knowledge homophily pattern in Large Language Models (LLMs)<n>We map LLM knowledge into a graph representation through knowledge checking at both the triplet and entity levels.<n>Motivated by this homophily principle, we propose a Graph Neural Network (GNN) regression model to estimate entity-level knowledgeability scores for triplets.
  arXiv  Detail & Related papers  (2025-09-28T09:40:27Z)
• Paths to Causality: Finding Informative Subgraphs Within Knowledge Graphs for Knowledge-Based Causal Discovery [10.573861741540853]
  We introduce a novel approach that integrates Knowledge Graphs (KGs) with Large Language Models (LLMs) to enhance knowledge-based causal discovery.<n>Our approach identifies informative metapath-based subgraphs within KGs and further refines the selection of these subgraphs using Learning-to-Rank-based models.<n>Our method outperforms most baselines by up to 44.4 points in F1 scores, evaluated across diverse LLMs and KGs.
  arXiv  Detail & Related papers  (2025-06-10T13:13:55Z)
• Harnessing Large Language Models for Knowledge Graph Question Answering via Adaptive Multi-Aspect Retrieval-Augmentation [81.18701211912779]
  We introduce an Adaptive Multi-Aspect Retrieval-augmented over KGs (Amar) framework.<n>This method retrieves knowledge including entities, relations, and subgraphs, and converts each piece of retrieved text into prompt embeddings.<n>Our method has achieved state-of-the-art performance on two common datasets.
  arXiv  Detail & Related papers  (2024-12-24T16:38:04Z)
• KaLM: Knowledge-aligned Autoregressive Language Modeling via Dual-view Knowledge Graph Contrastive Learning [74.21524111840652]
  This paper proposes textbfKaLM, a textitKnowledge-aligned Language Modeling approach.<n>It fine-tunes autoregressive large language models to align with KG knowledge via the joint objective of explicit knowledge alignment and implicit knowledge alignment.<n> Notably, our method achieves a significant performance boost in evaluations of knowledge-driven tasks.
  arXiv  Detail & Related papers  (2024-12-06T11:08:24Z)
• Learn to Unlearn: Meta-Learning-Based Knowledge Graph Embedding Unlearning [11.836591995678612]
  This paper introduces MetaEU, a Meta-Learning-Based Knowledge Graph Embedding Unlearning framework.<n> Experiments on benchmark datasets demonstrate its effectiveness in KG embedding unlearning.
  arXiv  Detail & Related papers  (2024-12-01T16:43:04Z)
• How Data Inter-connectivity Shapes LLMs Unlearning: A Structural Unlearning Perspective [29.924482732745954]
  Existing approaches assume data points to-be-forgotten are independent, ignoring their inter-connectivity.<n>We propose PISTOL, a method for compiling structural datasets.
  arXiv  Detail & Related papers  (2024-06-24T17:22:36Z)
• Recognizing Unseen Objects via Multimodal Intensive Knowledge Graph Propagation [68.13453771001522]
  We propose a multimodal intensive ZSL framework that matches regions of images with corresponding semantic embeddings. We conduct extensive experiments and evaluate our model on large-scale real-world data.
  arXiv  Detail & Related papers  (2023-06-14T13:07:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.