Efficient Knowledge Graph Unlearning with Zeroth-order Information

• URL: http://arxiv.org/abs/2508.14013v1
• Date: Tue, 19 Aug 2025 17:22:50 GMT
• Title: Efficient Knowledge Graph Unlearning with Zeroth-order Information
• Abstract summary: We present an efficient knowledge graph (KG) unlearning algorithm.<n> KG unlearning is nontrivial due to the distinctive structure of KG and the semantic relations between entities.<n>Our experimental results demonstrate that the proposed method outperforms other state-of-the-art graph unlearning baselines.
• Score: 16.462136830216856
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Due to regulations like the Right to be Forgotten, there is growing demand for removing training data and its influence from models. Since full retraining is costly, various machine unlearning methods have been proposed. In this paper, we firstly present an efficient knowledge graph (KG) unlearning algorithm. We remark that KG unlearning is nontrivial due to the distinctive structure of KG and the semantic relations between entities. Also, unlearning by estimating the influence of removed components incurs significant computational overhead when applied to large-scale knowledge graphs. To this end, we define an influence function for KG unlearning and propose to approximate the model's sensitivity without expensive computation of first-order and second-order derivatives for parameter updates. Specifically, we use Taylor expansion to estimate the parameter changes caused by data removal. Given that the first-order gradients and second-order derivatives dominate the computational load, we use the Fisher matrices and zeroth-order optimization to approximate the inverse-Hessian vector product without constructing the computational graphs. Our experimental results demonstrate that the proposed method outperforms other state-of-the-art graph unlearning baselines significantly in terms of unlearning efficiency and unlearning quality. Our code is released at https://github.com/NKUShaw/ZOWFKGIF.

Related papers

• Rethinking Efficient Graph Coarsening via a Non-Selfishness Principle [56.92868481531399]
  We propose a non-selfishness principle that prioritizes the collective interference of neighborhood in coarsening.<n>We derive a faster variant NOPE*, which reduces O(dot d) interference evaluation to O(d) based on the local isotropy assumption.<n>Learning on coarsened graphs yields comparable performance to original graphs, and can even show superior performance over LLM-based graph reasoning.
  arXiv  Detail & Related papers  (2026-05-13T05:24:35Z)
• Efficient Machine Unlearning via Influence Approximation [75.31015485113993]
  Influence-based unlearning has emerged as a prominent approach to estimate the impact of individual training samples on model parameters without retraining.<n>This paper establishes a theoretical link between memorizing (incremental learning) and forgetting (unlearning)<n>We introduce the Influence Approximation Unlearning algorithm for efficient machine unlearning from the incremental perspective.
  arXiv  Detail & Related papers  (2025-07-31T05:34:27Z)
• Knowledge Composition using Task Vectors with Learned Anisotropic Scaling [51.4661186662329]
  We introduce aTLAS, an algorithm that linearly combines parameter blocks with different learned coefficients, resulting in anisotropic scaling at the task vector level. We show that such linear combinations explicitly exploit the low intrinsicity of pre-trained models, with only a few coefficients being the learnable parameters. We demonstrate the effectiveness of our method in task arithmetic, few-shot recognition and test-time adaptation, with supervised or unsupervised objectives.
  arXiv  Detail & Related papers  (2024-07-03T07:54:08Z)
• Hessian-Free Online Certified Unlearning [8.875278412741695]
  We develop an online unlearning algorithm that achieves near-instantaneous data removal.<n>We prove that our proposed method outperforms the state-of-the-art methods in terms of the unlearning and generalization guarantees.
  arXiv  Detail & Related papers  (2024-04-02T07:54:18Z)
• Unifews: You Need Fewer Operations for Efficient Graph Neural Networks [9.66321358222326]
  Graph Neural Networks (GNNs) have shown promising performance, but at the cost of resource-intensive operations on graph-scale matrices.<n>We propose Unifews, a joint sparsification technique to unify graph and weight matrix operations and enhance GNN learning efficiency.
  arXiv  Detail & Related papers  (2024-03-20T03:07:30Z)
• DataInf: Efficiently Estimating Data Influence in LoRA-tuned LLMs and Diffusion Models [31.65198592956842]
  We propose DataInf, an efficient influence approximation method that is practical for large-scale generative AI models. Our theoretical analysis shows that DataInf is particularly well-suited for parameter-efficient fine-tuning techniques such as LoRA. In applications to RoBERTa-large, Llama-2-13B-chat, and stable-diffusion-v1.5 models, DataInf effectively identifies the most influential fine-tuning examples better than other approximate influence scores.
  arXiv  Detail & Related papers  (2023-10-02T04:59:19Z)
• GIF: A General Graph Unlearning Strategy via Influence Function [63.52038638220563]
  Graph Influence Function (GIF) is a model-agnostic unlearning method that can efficiently and accurately estimate parameter changes in response to a $epsilon$-mass perturbation in deleted data. We conduct extensive experiments on four representative GNN models and three benchmark datasets to justify GIF's superiority in terms of unlearning efficacy, model utility, and unlearning efficiency.
  arXiv  Detail & Related papers  (2023-04-06T03:02:54Z)
• Graph Polynomial Convolution Models for Node Classification of Non-Homophilous Graphs [52.52570805621925]
  We investigate efficient learning from higher-order graph convolution and learning directly from adjacency matrix for node classification. We show that the resulting model lead to new graphs and residual scaling parameter. We demonstrate that the proposed methods obtain improved accuracy for node-classification of non-homophilous parameters.
  arXiv  Detail & Related papers  (2022-09-12T04:46:55Z)
• Bringing Your Own View: Graph Contrastive Learning without Prefabricated Data Augmentations [94.41860307845812]
  Self-supervision is recently surging at its new frontier of graph learning. GraphCL uses a prefabricated prior reflected by the ad-hoc manual selection of graph data augmentations. We have extended the prefabricated discrete prior in the augmentation set, to a learnable continuous prior in the parameter space of graph generators. We have leveraged both principles of information minimization (InfoMin) and information bottleneck (InfoBN) to regularize the learned priors.
  arXiv  Detail & Related papers  (2022-01-04T15:49:18Z)
• Unrolling SGD: Understanding Factors Influencing Machine Unlearning [17.6607904333012]
  Machine unlearning is the process through which a deployed machine learning model forgets about one of its training data points. We first taxonomize approaches and metrics of approximate unlearning. We identify verification error, i.e., the L2 difference between the weights of an approximately unlearned and a naively retrained model.
  arXiv  Detail & Related papers  (2021-09-27T23:46:59Z)
• Machine Unlearning of Features and Labels [72.81914952849334]
  We propose first scenarios for unlearning and labels in machine learning models. Our approach builds on the concept of influence functions and realizes unlearning through closed-form updates of model parameters.
  arXiv  Detail & Related papers  (2021-08-26T04:42:24Z)
• An Advance on Variable Elimination with Applications to Tensor-Based Computation [11.358487655918676]
  We present new results on the classical algorithm of variable elimination, which underlies many algorithms including for probabilistic inference. The results relate to exploiting functional dependencies, allowing one to perform inference and learning efficiently on models that have very large treewidth.
  arXiv  Detail & Related papers  (2020-02-21T14:17:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.