Diffusion-based Hierarchical Negative Sampling for Multimodal Knowledge Graph Completion

• URL: http://arxiv.org/abs/2501.15393v1
• Date: Sun, 26 Jan 2025 04:20:34 GMT
• Title: Diffusion-based Hierarchical Negative Sampling for Multimodal Knowledge Graph Completion
• Authors: Guanglin Niu, Xiaowei Zhang,
• Abstract summary: Multimodal Knowledge Graph Completion (MMKGC) aims to address the critical issue of missing knowledge in multimodal knowledge graphs. Previous approaches ignore the employment of multimodal information to generate diverse and high-quality negative triples. We propose a novel Diffusion-based Hierarchical Negative Sampling scheme tailored for MMKGC tasks.
• Score: 6.24078177211832
• License:
• Abstract: Multimodal Knowledge Graph Completion (MMKGC) aims to address the critical issue of missing knowledge in multimodal knowledge graphs (MMKGs) for their better applications. However, both the previous MMGKC and negative sampling (NS) approaches ignore the employment of multimodal information to generate diverse and high-quality negative triples from various semantic levels and hardness levels, thereby limiting the effectiveness of training MMKGC models. Thus, we propose a novel Diffusion-based Hierarchical Negative Sampling (DHNS) scheme tailored for MMKGC tasks, which tackles the challenge of generating high-quality negative triples by leveraging a Diffusion-based Hierarchical Embedding Generation (DiffHEG) that progressively conditions on entities and relations as well as multimodal semantics. Furthermore, we develop a Negative Triple-Adaptive Training (NTAT) strategy that dynamically adjusts training margins associated with the hardness level of the synthesized negative triples, facilitating a more robust and effective learning procedure to distinguish between positive and negative triples. Extensive experiments on three MMKGC benchmark datasets demonstrate that our framework outperforms several state-of-the-art MMKGC models and negative sampling techniques, illustrating the effectiveness of our DHNS for training MMKGC models. The source codes and datasets of this paper are available at https://github.com/ngl567/DHNS.

Related papers

• Enhancing Unsupervised Sentence Embeddings via Knowledge-Driven Data Augmentation and Gaussian-Decayed Contrastive Learning [37.54523122932728]
  We propose a pipeline-based data augmentation method via large language models (LLMs) To tackle the issue of low data diversity, our pipeline utilizes knowledge graphs (KGs) to extract entities and quantities. To address high data noise, the GCSE model uses a Gaussian-decayed function to limit the impact of false hard negative samples.
  arXiv  Detail & Related papers  (2024-09-19T16:29:58Z)
• MMEvol: Empowering Multimodal Large Language Models with Evol-Instruct [148.39859547619156]
  We propose MMEvol, a novel multimodal instruction data evolution framework. MMEvol iteratively improves data quality through a refined combination of fine-grained perception, cognitive reasoning, and interaction evolution. Our approach reaches state-of-the-art (SOTA) performance in nine tasks using significantly less data compared to state-of-the-art models.
  arXiv  Detail & Related papers  (2024-09-09T17:44:00Z)
• PMT: Progressive Mean Teacher via Exploring Temporal Consistency for Semi-Supervised Medical Image Segmentation [51.509573838103854]
  We propose a semi-supervised learning framework, termed Progressive Mean Teachers (PMT), for medical image segmentation. Our PMT generates high-fidelity pseudo labels by learning robust and diverse features in the training process. Experimental results on two datasets with different modalities, i.e., CT and MRI, demonstrate that our method outperforms the state-of-the-art medical image segmentation approaches.
  arXiv  Detail & Related papers  (2024-09-08T15:02:25Z)
• Unleashing the Power of Imbalanced Modality Information for Multi-modal Knowledge Graph Completion [40.86196588992357]
  Multi-modal knowledge graph completion (MMKGC) aims to predict the missing triples in the multi-modal knowledge graphs. We propose Adaptive Multi-modal Fusion and Modality Adversarial Training (AdaMF-MAT) to unleash the power of imbalanced modality information. Our approach is a co-design of the MMKGC model and training strategy which can outperform 19 recent MMKGC methods.
  arXiv  Detail & Related papers  (2024-02-22T05:48:03Z)
• Learning Energy-Based Models by Cooperative Diffusion Recovery Likelihood [64.95663299945171]
  Training energy-based models (EBMs) on high-dimensional data can be both challenging and time-consuming. There exists a noticeable gap in sample quality between EBMs and other generative frameworks like GANs and diffusion models. We propose cooperative diffusion recovery likelihood (CDRL), an effective approach to tractably learn and sample from a series of EBMs.
  arXiv  Detail & Related papers  (2023-09-10T22:05:24Z)
• Knowledge Graph Completion with Pre-trained Multimodal Transformer and Twins Negative Sampling [13.016173217017597]
  We propose a VisualBERT-enhanced Knowledge Graph Completion model (VBKGC) for short. VBKGC could capture deeply fused multimodal information for entities and integrate them into the KGC model. We conduct extensive experiments to show the outstanding performance of VBKGC on the link prediction task.
  arXiv  Detail & Related papers  (2022-09-15T06:50:31Z)
• 3M: Multi-loss, Multi-path and Multi-level Neural Networks for speech recognition [31.992543274210835]
  We identify and integrate several approaches to achieve further improvements for ASR tasks. Specifically, multi-loss refers to the joint CTC/AED loss and multi-path denotes the Mixture-of-Experts(MoE) architecture. We evaluate our proposed method on the public WenetSpeech dataset and experimental results show that the proposed method provides 12.2%-17.6% relative CER improvement.
  arXiv  Detail & Related papers  (2022-04-07T03:10:49Z)
• Intermediate Layers Matter in Momentum Contrastive Self Supervised Learning [1.933681537640272]
  We show that bringing intermediate layers' representations of two augmented versions of an image closer together in self-supervised learning helps to improve the momentum contrastive (MoCo) method. We analyze the models trained using our novel approach via feature similarity analysis and layer-wise probing.
  arXiv  Detail & Related papers  (2021-10-27T22:40:41Z)
• Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learning [89.31889875864599]
  We propose an efficient model-based reinforcement learning algorithm for learning in multi-agent systems. Our main theoretical contributions are the first general regret bounds for model-based reinforcement learning for MFC. We provide a practical parametrization of the core optimization problem.
  arXiv  Detail & Related papers  (2021-07-08T18:01:02Z)
• Solving Inefficiency of Self-supervised Representation Learning [87.30876679780532]
  Existing contrastive learning methods suffer from very low learning efficiency. Under-clustering and over-clustering problems are major obstacles to learning efficiency. We propose a novel self-supervised learning framework using a median triplet loss.
  arXiv  Detail & Related papers  (2021-04-18T07:47:10Z)
• Reinforced Negative Sampling over Knowledge Graph for Recommendation [106.07209348727564]
  We develop a new negative sampling model, Knowledge Graph Policy Network (kgPolicy), which works as a reinforcement learning agent to explore high-quality negatives. kgPolicy navigates from the target positive interaction, adaptively receives knowledge-aware negative signals, and ultimately yields a potential negative item to train the recommender.
  arXiv  Detail & Related papers  (2020-03-12T12:44:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.