Curriculum Negative Mining For Temporal Networks

• URL: http://arxiv.org/abs/2407.17070v1
• Date: Wed, 24 Jul 2024 07:55:49 GMT
• Title: Curriculum Negative Mining For Temporal Networks
• Authors: Ziyue Chen, Tongya Zheng, Mingli Song,
• Abstract summary: Temporal networks are effective in capturing the evolving interactions of networks over time. CurNM is a model-aware curriculum learning framework that adaptively adjusts the difficulty of negative samples. Our method outperforms baseline methods by a significant margin.
• Score: 33.70909189731187
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Temporal networks are effective in capturing the evolving interactions of networks over time, such as social networks and e-commerce networks. In recent years, researchers have primarily concentrated on developing specific model architectures for Temporal Graph Neural Networks (TGNNs) in order to improve the representation quality of temporal nodes and edges. However, limited attention has been given to the quality of negative samples during the training of TGNNs. When compared with static networks, temporal networks present two specific challenges for negative sampling: positive sparsity and positive shift. Positive sparsity refers to the presence of a single positive sample amidst numerous negative samples at each timestamp, while positive shift relates to the variations in positive samples across different timestamps. To robustly address these challenges in training TGNNs, we introduce Curriculum Negative Mining (CurNM), a model-aware curriculum learning framework that adaptively adjusts the difficulty of negative samples. Within this framework, we first establish a dynamically updated negative pool that balances random, historical, and hard negatives to address the challenges posed by positive sparsity. Secondly, we implement a temporal-aware negative selection module that focuses on learning from the disentangled factors of recently active edges, thus accurately capturing shifting preferences. Extensive experiments on 12 datasets and 3 TGNNs demonstrate that our method outperforms baseline methods by a significant margin. Additionally, thorough ablation studies and parameter sensitivity experiments verify the usefulness and robustness of our approach. Our code is available at https://github.com/zziyue83/CurNM.

Related papers

• Robust Training of Temporal GNNs using Nearest Neighbours based Hard Negatives [44.49975766084011]
  Training of temporal graph neural networks Tgnn is enumerated by random sampling based unsupervised loss. We propose modified unsupervised learning of Tgnn, by replacing the uniform negative sampling with importance-based negative sampling. We show that Tgnn trained using loss based on proposed negative sampling provides consistent superior performance.
  arXiv  Detail & Related papers  (2024-02-14T15:27:53Z)
• Temporal graph models fail to capture global temporal dynamics [0.43512163406552007]
  We propose a trivial optimization-free baseline of "recently popular nodes" We show how standard negative sampling evaluation can be unsuitable for datasets with strong temporal dynamics. Our results indicate that temporal graph network architectures need deep rethinking for usage in problems with significant global dynamics.
  arXiv  Detail & Related papers  (2023-09-27T15:36:45Z)
• Evaluating Graph Neural Networks for Link Prediction: Current Pitfalls and New Benchmarking [66.83273589348758]
  Link prediction attempts to predict whether an unseen edge exists based on only a portion of edges of a graph. A flurry of methods have been introduced in recent years that attempt to make use of graph neural networks (GNNs) for this task. New and diverse datasets have also been created to better evaluate the effectiveness of these new models.
  arXiv  Detail & Related papers  (2023-06-18T01:58:59Z)
• Energy-based Out-of-Distribution Detection for Graph Neural Networks [76.0242218180483]
  We propose a simple, powerful and efficient OOD detection model for GNN-based learning on graphs, which we call GNNSafe. GNNSafe achieves up to $17.0%$ AUROC improvement over state-of-the-arts and it could serve as simple yet strong baselines in such an under-developed area.
  arXiv  Detail & Related papers  (2023-02-06T16:38:43Z)
• Link Prediction with Non-Contrastive Learning [19.340519670329382]
  Graph self-supervised learning (SSL) aims to derive useful node representations without labeled data. Many state-of-the-art graph SSL methods are contrastive methods, which use a combination of positive and negative samples. Recent literature introduced non-contrastive methods, which instead only use positive samples.
  arXiv  Detail & Related papers  (2022-11-25T22:30:15Z)
• Generating Negative Samples for Sequential Recommendation [83.60655196391855]
  We propose to Generate Negative Samples (items) for Sequential Recommendation (SR) A negative item is sampled at each time step based on the current SR model's learned user preferences toward items. Experiments on four public datasets verify the importance of providing high-quality negative samples for SR.
  arXiv  Detail & Related papers  (2022-08-07T05:44:13Z)
• Relation-aware Graph Attention Model With Adaptive Self-adversarial Training [29.240686573485718]
  This paper describes an end-to-end solution for the relationship prediction task in heterogeneous, multi-relational graphs. We particularly address two building blocks in the pipeline, namely heterogeneous graph representation learning and negative sampling. We introduce a parameter-free negative sampling technique -- adaptive self-adversarial (ASA) negative sampling.
  arXiv  Detail & Related papers  (2021-02-14T16:11:56Z)
• NPCFace: Negative-Positive Collaborative Training for Large-scale Face Recognition [78.21084529159577]
  We study how to make better use of hard samples for improving the training. The correlation between hard positive and hard negative is overlooked, and so is the relation between the margins in positive and negative logits. We propose a novel Negative-Positive Collaboration loss, named NPCFace, which emphasizes the training on both negative and positive hard cases.
  arXiv  Detail & Related papers  (2020-07-20T14:52:29Z)
• Understanding Negative Sampling in Graph Representation Learning [87.35038268508414]
  We show that negative sampling is as important as positive sampling in determining the optimization objective and the resulted variance. We propose Metropolis-Hastings (MCNS) to approximate the positive distribution with self-contrast approximation and accelerate negative sampling by Metropolis-Hastings. We evaluate our method on 5 datasets that cover extensive downstream graph learning tasks, including link prediction, node classification and personalized recommendation.
  arXiv  Detail & Related papers  (2020-05-20T06:25:21Z)
• Reinforced Negative Sampling over Knowledge Graph for Recommendation [106.07209348727564]
  We develop a new negative sampling model, Knowledge Graph Policy Network (kgPolicy), which works as a reinforcement learning agent to explore high-quality negatives. kgPolicy navigates from the target positive interaction, adaptively receives knowledge-aware negative signals, and ultimately yields a potential negative item to train the recommender.
  arXiv  Detail & Related papers  (2020-03-12T12:44:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.