Hard Sample Aware Network for Contrastive Deep Graph Clustering
- URL: http://arxiv.org/abs/2212.08665v1
- Date: Fri, 16 Dec 2022 16:57:37 GMT
- Title: Hard Sample Aware Network for Contrastive Deep Graph Clustering
- Authors: Yue Liu, Xihong Yang, Sihang Zhou, Xinwang Liu, Zhen Wang, Ke Liang,
Wenxuan Tu, Liang Li, Jingcan Duan, Cancan Chen
- Abstract summary: We propose a novel contrastive deep graph clustering method dubbed Hard Sample Aware Network (HSAN)
In our algorithm, the similarities between samples are calculated by considering both the attribute embeddings and the structure embeddings.
Under the guidance of the carefully collected high-confidence clustering information, our proposed weight modulating function will first recognize the positive and negative samples.
- Score: 38.44763843990694
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Contrastive deep graph clustering, which aims to divide nodes into disjoint
groups via contrastive mechanisms, is a challenging research spot. Among the
recent works, hard sample mining-based algorithms have achieved great attention
for their promising performance. However, we find that the existing hard sample
mining methods have two problems as follows. 1) In the hardness measurement,
the important structural information is overlooked for similarity calculation,
degrading the representativeness of the selected hard negative samples. 2)
Previous works merely focus on the hard negative sample pairs while neglecting
the hard positive sample pairs. Nevertheless, samples within the same cluster
but with low similarity should also be carefully learned. To solve the
problems, we propose a novel contrastive deep graph clustering method dubbed
Hard Sample Aware Network (HSAN) by introducing a comprehensive similarity
measure criterion and a general dynamic sample weighing strategy. Concretely,
in our algorithm, the similarities between samples are calculated by
considering both the attribute embeddings and the structure embeddings, better
revealing sample relationships and assisting hardness measurement. Moreover,
under the guidance of the carefully collected high-confidence clustering
information, our proposed weight modulating function will first recognize the
positive and negative samples and then dynamically up-weight the hard sample
pairs while down-weighting the easy ones. In this way, our method can mine not
only the hard negative samples but also the hard positive sample, thus
improving the discriminative capability of the samples further. Extensive
experiments and analyses demonstrate the superiority and effectiveness of our
proposed method.
Related papers
- Synthetic Hard Negative Samples for Contrastive Learning [8.776888865665024]
This paper proposes a novel feature-level method, namely sampling synthetic hard negative samples for contrastive learning (SSCL)
We generate more and harder negative samples by mixing negative samples, and then sample them by controlling the contrast of anchor sample with the other negative samples.
Our proposed method improves the classification performance on different image datasets and can be readily integrated into existing methods.
arXiv Detail & Related papers (2023-04-06T09:54:35Z) - Self-supervised Training Sample Difficulty Balancing for Local
Descriptor Learning [1.309716118537215]
In the case of an imbalance between positive and negative samples, hard negative mining strategies have been shown to help models learn more subtle differences.
However, if too strict mining strategies are promoted in the dataset, there may be a risk of introducing false negative samples.
In this paper, we investigate how to trade off the difficulty of the mined samples in order to obtain and exploit high-quality negative samples.
arXiv Detail & Related papers (2023-03-10T18:37:43Z) - Cluster-guided Contrastive Graph Clustering Network [53.16233290797777]
We propose a Cluster-guided Contrastive deep Graph Clustering network (CCGC)
We construct two views of the graph by designing special Siamese encoders whose weights are not shared between the sibling sub-networks.
To construct semantic meaningful negative sample pairs, we regard the centers of different high-confidence clusters as negative samples.
arXiv Detail & Related papers (2023-01-03T13:42:38Z) - Saliency Grafting: Innocuous Attribution-Guided Mixup with Calibrated
Label Mixing [104.630875328668]
Mixup scheme suggests mixing a pair of samples to create an augmented training sample.
We present a novel, yet simple Mixup-variant that captures the best of both worlds.
arXiv Detail & Related papers (2021-12-16T11:27:48Z) - Density-Based Dynamic Curriculum Learning for Intent Detection [14.653917644725427]
Our model defines the sample's difficulty level according to their eigenvectors' density.
We apply a dynamic curriculum learning strategy, which pays distinct attention to samples of various difficulty levels.
Experiments on three open datasets verify that the proposed density-based algorithm can distinguish simple and complex samples significantly.
arXiv Detail & Related papers (2021-08-24T12:29:26Z) - Jo-SRC: A Contrastive Approach for Combating Noisy Labels [58.867237220886885]
We propose a noise-robust approach named Jo-SRC (Joint Sample Selection and Model Regularization based on Consistency)
Specifically, we train the network in a contrastive learning manner. Predictions from two different views of each sample are used to estimate its "likelihood" of being clean or out-of-distribution.
arXiv Detail & Related papers (2021-03-24T07:26:07Z) - Doubly Contrastive Deep Clustering [135.7001508427597]
We present a novel Doubly Contrastive Deep Clustering (DCDC) framework, which constructs contrastive loss over both sample and class views.
Specifically, for the sample view, we set the class distribution of the original sample and its augmented version as positive sample pairs.
For the class view, we build the positive and negative pairs from the sample distribution of the class.
In this way, two contrastive losses successfully constrain the clustering results of mini-batch samples in both sample and class level.
arXiv Detail & Related papers (2021-03-09T15:15:32Z) - The Simulator: Understanding Adaptive Sampling in the
Moderate-Confidence Regime [52.38455827779212]
We propose a novel technique for analyzing adaptive sampling called the em Simulator.
We prove the first instance-based lower bounds the top-k problem which incorporate the appropriate log-factors.
Our new analysis inspires a simple and near-optimal for the best-arm and top-k identification, the first em practical of its kind for the latter problem.
arXiv Detail & Related papers (2017-02-16T23:42:02Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.