When Modalities Remember: Continual Learning for Multimodal Knowledge Graphs

• URL: http://arxiv.org/abs/2604.02778v1
• Date: Fri, 03 Apr 2026 06:40:04 GMT
• Title: When Modalities Remember: Continual Learning for Multimodal Knowledge Graphs
• Abstract summary: Real-world multimodal knowledge graphs (MMKGs) are dynamic, with new entities, relations, and multimodal knowledge emerging over time.<n>Existing continual knowledge graph reasoning (CKGR) methods focus on structural triples and cannot fully exploit multimodal signals from new entities.<n>We present a systematic study of continual multimodal knowledge graph reasoning (CMMKGR)<n>It employs a multimodal-structural collaborative curriculum to schedule progressive learning based on the structural connectivity of new triples to the historical graph.
• Score: 37.80751349022448
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Real-world multimodal knowledge graphs (MMKGs) are dynamic, with new entities, relations, and multimodal knowledge emerging over time. Existing continual knowledge graph reasoning (CKGR) methods focus on structural triples and cannot fully exploit multimodal signals from new entities. Existing multimodal knowledge graph reasoning (MMKGR) methods, however, usually assume static graphs and suffer catastrophic forgetting as graphs evolve. To address this gap, we present a systematic study of continual multimodal knowledge graph reasoning (CMMKGR). We construct several continual multimodal knowledge graph benchmarks from existing MMKG datasets and propose MRCKG, a new CMMKGR model. Specifically, MRCKG employs a multimodal-structural collaborative curriculum to schedule progressive learning based on the structural connectivity of new triples to the historical graph and their multimodal compatibility. It also introduces a cross-modal knowledge preservation mechanism to mitigate forgetting through entity representation stability, relational semantic consistency, and modality anchoring. In addition, a multimodal contrastive replay scheme with a two-stage optimization strategy reinforces learned knowledge via multimodal importance sampling and representation alignment. Experiments on multiple datasets show that MRCKG preserves previously learned multimodal knowledge while substantially improving the learning of new knowledge.

Related papers

• ReCoG: Reciprocal Co-Evolution for Multimodal Graph Learning [9.030717765733398]
  Multimodal graph learning requires jointly training over graph structure and heterogeneous node attributes.<n>We propose ReCoG, a new learning paradigm that tightly couples graph structure learning and multimodal representation learning.<n>Our results demonstrate that reciprocal co-evolution of structure and semantics is important for effective multimodal graph learning.
  arXiv  Detail & Related papers  (2026-08-24T04:16:13Z)
• VL-KGE: Vision-Language Models Meet Knowledge Graph Embeddings [14.36934698922473]
  Vision-Language Models (VLMs) offer a powerful way to align diverse modalities within a shared embedding space.<n>We propose Vision-Language Knowledge Graph Embeddings (VL-KGE), a framework that integrates cross-modal alignment from VLMs with structured relational modeling.
  arXiv  Detail & Related papers  (2026-03-02T22:18:48Z)
• Graph4MM: Weaving Multimodal Learning with Structural Information [52.16646463590474]
  Graphs provide powerful structural information for modeling intra- and inter-modal relationships.<n>Previous works fail to distinguish multi-hop neighbors and treat the graph as a standalone modality.<n>We propose Graph4MM, a graph-based multimodal learning framework.
  arXiv  Detail & Related papers  (2025-10-19T20:13:03Z)
• VAT-KG: Knowledge-Intensive Multimodal Knowledge Graph Dataset for Retrieval-Augmented Generation [16.248703946640735]
  Multimodal Knowledge Graphs (MMKGs) represent explicit knowledge across multiple modalities.<n>Visual-Audio-Text Knowledge Graph (VAT-KG) is first concept-centric and knowledge-intensive multimodal knowledge graph.
  arXiv  Detail & Related papers  (2025-06-11T07:22:57Z)
• Multiple Heads are Better than One: Mixture of Modality Knowledge Experts for Entity Representation Learning [51.80447197290866]
  Learning high-quality multi-modal entity representations is an important goal of multi-modal knowledge graph (MMKG) representation learning.<n>Existing methods focus on crafting elegant entity-wise multi-modal fusion strategies.<n>We introduce a novel framework with Mixture of Modality Knowledge experts (MoMoK) to learn adaptive multi-modal entity representations.
  arXiv  Detail & Related papers  (2024-05-27T06:36:17Z)
• Tokenization, Fusion, and Augmentation: Towards Fine-grained Multi-modal Entity Representation [51.80447197290866]
  Multi-modal knowledge graph completion (MMKGC) aims to discover unobserved knowledge from given knowledge graphs.<n>Existing MMKGC methods usually extract multi-modal features with pre-trained models.<n>We introduce a novel framework MyGO to tokenize, fuse, and augment the fine-grained multi-modal representations of entities.
  arXiv  Detail & Related papers  (2024-04-15T05:40:41Z)
• Zero-Shot Relational Learning for Multimodal Knowledge Graphs [31.215889061734295]
  We propose a novel end-to-end framework, consisting of three components, i.e., multimodal learner, structure consolidator embedding generator, to integrate diverse multimodal information and knowledge graph structures.<n> Evaluation results on three multimodal knowledge graphs demonstrate the superior performance of our proposed method.
  arXiv  Detail & Related papers  (2024-04-09T11:14:45Z)
• Continual Multimodal Knowledge Graph Construction [62.77243705682985]
  Current Multimodal Knowledge Graph Construction (MKGC) models struggle with the real-world dynamism of continuously emerging entities and relations. This study introduces benchmarks aimed at fostering the development of the continual MKGC domain. We introduce MSPT framework, designed to surmount the shortcomings of existing MKGC approaches during multimedia data processing.
  arXiv  Detail & Related papers  (2023-05-15T14:58:28Z)
• A Unified Continuous Learning Framework for Multi-modal Knowledge Discovery and Pre-training [73.7507857547549]
  We propose to unify knowledge discovery and multi-modal pre-training in a continuous learning framework. For knowledge discovery, a pre-trained model is used to identify cross-modal links on a graph. For model pre-training, the knowledge graph is used as the external knowledge to guide the model updating.
  arXiv  Detail & Related papers  (2022-06-11T16:05:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.