MixKG: Mixing for harder negative samples in knowledge graph
- URL: http://arxiv.org/abs/2202.09606v1
- Date: Sat, 19 Feb 2022 13:31:06 GMT
- Title: MixKG: Mixing for harder negative samples in knowledge graph
- Authors: Feihu Che, Guohua Yang, Pengpeng Shao, Dawei Zhang, Jianhua Tao
- Abstract summary: Knowledge graph embedding(KGE) aims to represent entities and relations into low-dimensional vectors for many real-world applications.
We introduce an inexpensive but effective method called MixKG to generate harder negative samples for knowledge graphs.
Experiments on two public datasets and four classical KGE methods show MixKG is superior to previous negative sampling algorithms.
- Score: 33.4379457065033
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Knowledge graph embedding~(KGE) aims to represent entities and relations into
low-dimensional vectors for many real-world applications. The representations
of entities and relations are learned via contrasting the positive and negative
triplets. Thus, high-quality negative samples are extremely important in KGE.
However, the present KGE models either rely on simple negative sampling
methods, which makes it difficult to obtain informative negative triplets; or
employ complex adversarial methods, which requires more training data and
strategies. In addition, these methods can only construct negative triplets
using the existing entities, which limits the potential to explore harder
negative triplets. To address these issues, we adopt mixing operation in
generating harder negative samples for knowledge graphs and introduce an
inexpensive but effective method called MixKG. Technically, MixKG first
proposes two kinds of criteria to filter hard negative triplets among the
sampled negatives: based on scoring function and based on correct entity
similarity. Then, MixKG synthesizes harder negative samples via the convex
combinations of the paired selected hard negatives. Experiments on two public
datasets and four classical KGE methods show MixKG is superior to previous
negative sampling algorithms.
Related papers
- From Overfitting to Robustness: Quantity, Quality, and Variety Oriented Negative Sample Selection in Graph Contrastive Learning [38.87932592059369]
Graph contrastive learning (GCL) aims to contrast positive-negative counterparts to learn the node embeddings.
The variation, quantity, and quality of negative samples compared to positive samples play crucial roles in learning meaningful embeddings for node classification downstream tasks.
This study proposes a novel Cumulative Sample Selection (CSS) algorithm by comprehensively considering negative samples' quality, variations, and quantity.
arXiv Detail & Related papers (2024-06-21T10:47:26Z) - CKD: Contrastive Knowledge Distillation from A Sample-wise Perspective [48.99488315273868]
We present a contrastive knowledge distillation approach, which can be formulated as a sample-wise alignment problem with intra- and inter-sample constraints.
Our method minimizes logit differences within the same sample by considering their numerical values.
We conduct comprehensive experiments on three datasets including CIFAR-100, ImageNet-1K, and MS COCO.
arXiv Detail & Related papers (2024-04-22T11:52:40Z) - Contrastive Learning with Negative Sampling Correction [52.990001829393506]
We propose a novel contrastive learning method named Positive-Unlabeled Contrastive Learning (PUCL)
PUCL treats the generated negative samples as unlabeled samples and uses information from positive samples to correct bias in contrastive loss.
PUCL can be applied to general contrastive learning problems and outperforms state-of-the-art methods on various image and graph classification tasks.
arXiv Detail & Related papers (2024-01-13T11:18:18Z) - Negative Sampling with Adaptive Denoising Mixup for Knowledge Graph
Embedding [36.24764655034505]
Knowledge graph embedding (KGE) aims to map entities and relations of a knowledge graph (KG) into a low-dimensional and dense vector space via contrasting the positive and negative triples.
Negative sampling is essential to find high-quality negative triples since KGs only contain positive triples.
Most existing negative sampling methods assume that non-existent triples with high scores are high-quality negative triples.
arXiv Detail & Related papers (2023-10-15T09:01:24Z) - DropMix: Better Graph Contrastive Learning with Harder Negative Samples [6.242575753642188]
Mixup has been introduced to synthesize hard negative samples in graph contrastive learning (GCL)
We propose a novel method DropMix to synthesize harder negative samples.
arXiv Detail & Related papers (2023-10-15T07:45:30Z) - Your Negative May not Be True Negative: Boosting Image-Text Matching
with False Negative Elimination [62.18768931714238]
We propose a novel False Negative Elimination (FNE) strategy to select negatives via sampling.
The results demonstrate the superiority of our proposed false negative elimination strategy.
arXiv Detail & Related papers (2023-08-08T16:31:43Z) - Entity Aware Negative Sampling with Auxiliary Loss of False Negative
Prediction for Knowledge Graph Embedding [0.0]
We propose a novel method called Entity Aware Negative Sampling (EANS)
EANS is able to sample negative entities resemble to positive one by adopting Gaussian distribution to the aligned entity index space.
The proposed method can generate high-quality negative samples regardless of negative sample size and effectively mitigate the influence of false negative samples.
arXiv Detail & Related papers (2022-10-12T14:27:51Z) - Language Model-driven Negative Sampling [8.299192665823542]
Knowledge Graph Embeddings (KGEs) encode the entities and relations of a knowledge graph (KG) into a vector space with a purpose of representation learning and reasoning for an ultimate downstream task (i.e., link prediction, question answering)
Since KGEs follow closed-world assumption and assume all the present facts in KGs to be positive (correct), they also require negative samples as a counterpart for learning process for truthfulness test of existing triples.
We propose an approach for generating negative sampling considering the existing rich textual knowledge in KGs.
arXiv Detail & Related papers (2022-03-09T13:27:47Z) - Doubly Contrastive Deep Clustering [135.7001508427597]
We present a novel Doubly Contrastive Deep Clustering (DCDC) framework, which constructs contrastive loss over both sample and class views.
Specifically, for the sample view, we set the class distribution of the original sample and its augmented version as positive sample pairs.
For the class view, we build the positive and negative pairs from the sample distribution of the class.
In this way, two contrastive losses successfully constrain the clustering results of mini-batch samples in both sample and class level.
arXiv Detail & Related papers (2021-03-09T15:15:32Z) - Contrastive Learning with Hard Negative Samples [80.12117639845678]
We develop a new family of unsupervised sampling methods for selecting hard negative samples.
A limiting case of this sampling results in a representation that tightly clusters each class, and pushes different classes as far apart as possible.
The proposed method improves downstream performance across multiple modalities, requires only few additional lines of code to implement, and introduces no computational overhead.
arXiv Detail & Related papers (2020-10-09T14:18:53Z) - Reinforced Negative Sampling over Knowledge Graph for Recommendation [106.07209348727564]
We develop a new negative sampling model, Knowledge Graph Policy Network (kgPolicy), which works as a reinforcement learning agent to explore high-quality negatives.
kgPolicy navigates from the target positive interaction, adaptively receives knowledge-aware negative signals, and ultimately yields a potential negative item to train the recommender.
arXiv Detail & Related papers (2020-03-12T12:44:30Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.