Modality-Aware Negative Sampling for Multi-modal Knowledge Graph Embedding

• URL: http://arxiv.org/abs/2304.11618v1
• Date: Sun, 23 Apr 2023 11:22:17 GMT
• Title: Modality-Aware Negative Sampling for Multi-modal Knowledge Graph Embedding
• Authors: Yichi Zhang, Mingyang Chen, Wen Zhang
• Abstract summary: Negative sampling (NS) is widely used in knowledge graph embedding (KGE), which aims to generate negative triples to make a positive-negative contrast during training. Existing NS methods are unsuitable when multi-modal information is considered in KGE models. We propose Modality-Aware Negative Sampling (MANS) for multi-modal knowledge graph embedding (MMKGE) to address the mentioned problems.
• Score: 12.513266782679754
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Negative sampling (NS) is widely used in knowledge graph embedding (KGE), which aims to generate negative triples to make a positive-negative contrast during training. However, existing NS methods are unsuitable when multi-modal information is considered in KGE models. They are also inefficient due to their complex design. In this paper, we propose Modality-Aware Negative Sampling (MANS) for multi-modal knowledge graph embedding (MMKGE) to address the mentioned problems. MANS could align structural and visual embeddings for entities in KGs and learn meaningful embeddings to perform better in multi-modal KGE while keeping lightweight and efficient. Empirical results on two benchmarks demonstrate that MANS outperforms existing NS methods. Meanwhile, we make further explorations about MANS to confirm its effectiveness.

Related papers

• Negative Sampling in Knowledge Graph Representation Learning: A Review [2.6703221234079946]
  Knowledge Graph Representation Learning (KGRL) is essential for AI applications like knowledge construction and information retrieval. generating high-quality negative samples from existing knowledge graphs is challenging. This paper systematically reviews various negative sampling (NS) methods and their contributions to the success of KGRL.
  arXiv  Detail & Related papers  (2024-02-29T14:26:20Z)
• Self-Supervised Neuron Segmentation with Multi-Agent Reinforcement Learning [53.00683059396803]
  Mask image model (MIM) has been widely used due to its simplicity and effectiveness in recovering original information from masked images. We propose a decision-based MIM that utilizes reinforcement learning (RL) to automatically search for optimal image masking ratio and masking strategy. Our approach has a significant advantage over alternative self-supervised methods on the task of neuron segmentation.
  arXiv  Detail & Related papers  (2023-10-06T10:40:46Z)
• BroadCAM: Outcome-agnostic Class Activation Mapping for Small-scale Weakly Supervised Applications [69.22739434619531]
  We propose an outcome-agnostic CAM approach, called BroadCAM, for small-scale weakly supervised applications. By evaluating BroadCAM on VOC2012 and BCSS-WSSS for WSSS and OpenImages30k for WSOL, BroadCAM demonstrates superior performance.
  arXiv  Detail & Related papers  (2023-09-07T06:45:43Z)
• Multimodal Contrastive Learning with Hard Negative Sampling for Human Activity Recognition [14.88934924520362]
  Human Activity Recognition (HAR) systems have been extensively studied by the vision and ubiquitous computing communities. We propose a hard negative sampling method for multimodal HAR with a hard negative sampling loss for skeleton and IMU data pairs. We demonstrate the robustness of our approach forlearning strong feature representation for HAR tasks, and on the limited data setting.
  arXiv  Detail & Related papers  (2023-09-03T20:00:37Z)
• Neighborhood-based Hard Negative Mining for Sequential Recommendation [14.66576013324401]
  Negative sampling plays a crucial role in training successful sequential recommendation models. We propose a Graph-based Negative sampling approach based on Neighborhood Overlap (GNNO) to exploit structural information hidden in user behaviors for negative mining.
  arXiv  Detail & Related papers  (2023-06-12T12:28:54Z)
• Knowledge Graph Completion with Pre-trained Multimodal Transformer and Twins Negative Sampling [13.016173217017597]
  We propose a VisualBERT-enhanced Knowledge Graph Completion model (VBKGC) for short. VBKGC could capture deeply fused multimodal information for entities and integrate them into the KGC model. We conduct extensive experiments to show the outstanding performance of VBKGC on the link prediction task.
  arXiv  Detail & Related papers  (2022-09-15T06:50:31Z)
• Absolute Wrong Makes Better: Boosting Weakly Supervised Object Detection via Negative Deterministic Information [54.35679298764169]
  Weakly supervised object detection (WSOD) is a challenging task, in which image-level labels are used to train an object detector. This paper focuses on identifying and fully exploiting the deterministic information in WSOD. We propose a negative deterministic information (NDI) based method for improving WSOD, namely NDI-WSOD.
  arXiv  Detail & Related papers  (2022-04-21T12:55:27Z)
• DSKReG: Differentiable Sampling on Knowledge Graph for Recommendation with Relational GNN [59.160401038969795]
  We propose differentiable sampling on Knowledge Graph for Recommendation with GNN (DSKReG) We devise a differentiable sampling strategy, which enables the selection of relevant items to be jointly optimized with the model training procedure. The experimental results demonstrate that our model outperforms state-of-the-art KG-based recommender systems.
  arXiv  Detail & Related papers  (2021-08-26T16:19:59Z)
• KDExplainer: A Task-oriented Attention Model for Explaining Knowledge Distillation [59.061835562314066]
  We introduce a novel task-oriented attention model, termed as KDExplainer, to shed light on the working mechanism underlying the vanilla KD. We also introduce a portable tool, dubbed as virtual attention module (VAM), that can be seamlessly integrated with various deep neural networks (DNNs) to enhance their performance under KD.
  arXiv  Detail & Related papers  (2021-05-10T08:15:26Z)
• Reinforced Negative Sampling over Knowledge Graph for Recommendation [106.07209348727564]
  We develop a new negative sampling model, Knowledge Graph Policy Network (kgPolicy), which works as a reinforcement learning agent to explore high-quality negatives. kgPolicy navigates from the target positive interaction, adaptively receives knowledge-aware negative signals, and ultimately yields a potential negative item to train the recommender.
  arXiv  Detail & Related papers  (2020-03-12T12:44:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.