Improving Continual Learning of Knowledge Graph Embeddings via Informed Initialization

• URL: http://arxiv.org/abs/2511.11118v1
• Date: Fri, 14 Nov 2025 09:48:24 GMT
• Title: Improving Continual Learning of Knowledge Graph Embeddings via Informed Initialization
• Abstract summary: Many Knowledege Graphs (KGs) are frequently updated, forcing their Knowledge Graph Embeddings (KGEs) to adapt to these changes.<n>To address this problem, continual learning techniques for KGEs incorporate embeddings for new entities while updating the old ones.<n>We propose a novel informed embedding strategy, which can be seamlessly integrated into existing continual learning methods for KGE.
• Score: 0.8602553195689513
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Many Knowledege Graphs (KGs) are frequently updated, forcing their Knowledge Graph Embeddings (KGEs) to adapt to these changes. To address this problem, continual learning techniques for KGEs incorporate embeddings for new entities while updating the old ones. One necessary step in these methods is the initialization of the embeddings, as an input to the KGE learning process, which can have an important impact in the accuracy of the final embeddings, as well as in the time required to train them. This is especially relevant for relatively small and frequent updates. We propose a novel informed embedding initialization strategy, which can be seamlessly integrated into existing continual learning methods for KGE, that enhances the acquisition of new knowledge while reducing catastrophic forgetting. Specifically, the KG schema and the previously learned embeddings are utilized to obtain initial representations for the new entities, based on the classes the entities belong to. Our extensive experimental analysis shows that the proposed initialization strategy improves the predictive performance of the resulting KGEs, while also enhancing knowledge retention. Furthermore, our approach accelerates knowledge acquisition, reducing the number of epochs, and therefore time, required to incrementally learn new embeddings. Finally, its benefits across various types of KGE learning models are demonstrated.

Related papers

• AgREE: Agentic Reasoning for Knowledge Graph Completion on Emerging Entities [16.03316404374351]
  Agentic Reasoning for Emerging Entities (AgREE) is a novel framework that combines iterative retrieval actions and multi-step reasoning to dynamically construct rich knowledge graph triplets.<n>AgREE significantly outperforms existing methods in constructing knowledge graph triplets, especially for emerging entities that were not seen during language models' training processes.<n>Our work demonstrates the effectiveness of combining agent-based reasoning with strategic information retrieval for maintaining up-to-date knowledge graphs.
  arXiv  Detail & Related papers  (2025-08-06T06:34:22Z)
• Knowledge Graph Enhanced Generative Multi-modal Models for Class-Incremental Learning [51.0864247376786]
  We introduce a Knowledge Graph Enhanced Generative Multi-modal model (KG-GMM) that builds an evolving knowledge graph throughout the learning process.<n>During testing, we propose a Knowledge Graph Augmented Inference method that locates specific categories by analyzing relationships within the generated text.
  arXiv  Detail & Related papers  (2025-03-24T07:20:43Z)
• A Good Start Matters: Enhancing Continual Learning with Data-Driven Weight Initialization [15.8696301825572]
  Continuously-trained deep neural networks (DNNs) must rapidly learn new concepts while preserving and utilizing prior knowledge.<n>Weights for newly encountered categories are typically randomly, leading to high initial training loss (spikes) and instability.<n>Inspired by Neural Collapse (NC), we propose a weight initialization strategy to improve learning efficiency in CL.
  arXiv  Detail & Related papers  (2025-03-09T01:44:22Z)
• How Do LLMs Acquire New Knowledge? A Knowledge Circuits Perspective on Continual Pre-Training [92.88889953768455]
  Large Language Models (LLMs) face a critical gap in understanding how they internalize new knowledge.<n>We identify computational subgraphs that facilitate knowledge storage and processing.
  arXiv  Detail & Related papers  (2025-02-16T16:55:43Z)
• Towards Continual Knowledge Graph Embedding via Incremental Distillation [12.556752486002356]
  Traditional knowledge graph embedding (KGE) methods typically require preserving the entire knowledge graph (KG) with significant training costs when new knowledge emerges. This paper proposes a competitive method for CKGE based on incremental distillation (IncDE), which considers the full use of the explicit graph structure in KGs.
  arXiv  Detail & Related papers  (2024-05-07T16:16:00Z)
• A Unified and General Framework for Continual Learning [58.72671755989431]
  Continual Learning (CL) focuses on learning from dynamic and changing data distributions while retaining previously acquired knowledge. Various methods have been developed to address the challenge of catastrophic forgetting, including regularization-based, Bayesian-based, and memory-replay-based techniques. This research aims to bridge this gap by introducing a comprehensive and overarching framework that encompasses and reconciles these existing methodologies.
  arXiv  Detail & Related papers  (2024-03-20T02:21:44Z)
• Knowledge Editing for Large Language Models: A Survey [51.01368551235289]
  One major drawback of large language models (LLMs) is their substantial computational cost for pre-training. Knowledge-based Model Editing (KME) has attracted increasing attention, which aims to precisely modify the LLMs to incorporate specific knowledge.
  arXiv  Detail & Related papers  (2023-10-24T22:18:13Z)
• Overcoming Generic Knowledge Loss with Selective Parameter Update [48.240683797965005]
  We propose a novel approach to continuously update foundation models. Instead of updating all parameters equally, we localize the updates to a sparse set of parameters relevant to the task being learned. Our method achieves improvements on the accuracy of the newly learned tasks up to 7% while preserving the pretraining knowledge with a negligible decrease of 0.9% on a representative control set accuracy.
  arXiv  Detail & Related papers  (2023-08-23T22:55:45Z)
• Confidence-aware Self-Semantic Distillation on Knowledge Graph Embedding [20.49583906923656]
  Confidence-aware Self-Knowledge Distillation learns from the model itself to enhance KGE in a low-dimensional space. A specific semantic module is developed to filter reliable knowledge by estimating the confidence of previously learned embeddings.
  arXiv  Detail & Related papers  (2022-06-07T01:49:22Z)
• New Insights on Reducing Abrupt Representation Change in Online Continual Learning [69.05515249097208]
  We focus on the change in representations of observed data that arises when previously unobserved classes appear in the incoming data stream. We show that applying Experience Replay causes the newly added classes' representations to overlap significantly with the previous classes. We propose a new method which mitigates this issue by shielding the learned representations from drastic adaptation to accommodate new classes.
  arXiv  Detail & Related papers  (2022-03-08T01:37:00Z)
• Generative Adversarial Zero-Shot Relational Learning for Knowledge Graphs [96.73259297063619]
  We consider a novel formulation, zero-shot learning, to free this cumbersome curation. For newly-added relations, we attempt to learn their semantic features from their text descriptions. We leverage Generative Adrial Networks (GANs) to establish the connection between text and knowledge graph domain.
  arXiv  Detail & Related papers  (2020-01-08T01:19:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.