Exploring the Performance of Continuous-Time Dynamic Link Prediction Algorithms

• URL: http://arxiv.org/abs/2405.17182v1
• Date: Mon, 27 May 2024 14:03:28 GMT
• Title: Exploring the Performance of Continuous-Time Dynamic Link Prediction Algorithms
• Authors: Raphaël Romero, Maarten Buyl, Tijl De Bie, Jefrey Lijffijt,
• Abstract summary: Dynamic Link Prediction (DLP) addresses the prediction of future links in evolving networks. In this work, we contribute tools to perform such a comprehensive evaluation. We describe an exhaustive taxonomy of negative sampling methods that can be used at evaluation time.
• Score: 14.82820088479196
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Dynamic Link Prediction (DLP) addresses the prediction of future links in evolving networks. However, accurately portraying the performance of DLP algorithms poses challenges that might impede progress in the field. Importantly, common evaluation pipelines usually calculate ranking or binary classification metrics, where the scores of observed interactions (positives) are compared with those of randomly generated ones (negatives). However, a single metric is not sufficient to fully capture the differences between DLP algorithms, and is prone to overly optimistic performance evaluation. Instead, an in-depth evaluation should reflect performance variations across different nodes, edges, and time segments. In this work, we contribute tools to perform such a comprehensive evaluation. (1) We propose Birth-Death diagrams, a simple but powerful visualization technique that illustrates the effect of time-based train-test splitting on the difficulty of DLP on a given dataset. (2) We describe an exhaustive taxonomy of negative sampling methods that can be used at evaluation time. (3) We carry out an empirical study of the effect of the different negative sampling strategies. Our comparison between heuristics and state-of-the-art memory-based methods on various real-world datasets confirms a strong effect of using different negative sampling strategies on the test Area Under the Curve (AUC). Moreover, we conduct a visual exploration of the prediction, with additional insights on which different types of errors are prominent over time.

Related papers

• Bidirectional Decoding: Improving Action Chunking via Closed-Loop Resampling [51.38330727868982]
  Bidirectional Decoding (BID) is a test-time inference algorithm that bridges action chunking with closed-loop operations. We show that BID boosts the performance of two state-of-the-art generative policies across seven simulation benchmarks and two real-world tasks.
  arXiv  Detail & Related papers  (2024-08-30T15:39:34Z)
• New Perspectives on the Evaluation of Link Prediction Algorithms for Dynamic Graphs [12.987894327817159]
  We introduce novel visualization methods that can yield insight into prediction performance and the dynamics of temporal networks. We validate empirically, on datasets extracted from recent benchmarks, that the error is typically not evenly distributed across different data segments.
  arXiv  Detail & Related papers  (2023-11-30T11:57:07Z)
• Time-series Generation by Contrastive Imitation [87.51882102248395]
  We study a generative framework that seeks to combine the strengths of both: Motivated by a moment-matching objective to mitigate compounding error, we optimize a local (but forward-looking) transition policy. At inference, the learned policy serves as the generator for iterative sampling, and the learned energy serves as a trajectory-level measure for evaluating sample quality.
  arXiv  Detail & Related papers  (2023-11-02T16:45:25Z)
• Multi-Task Self-Supervised Time-Series Representation Learning [3.31490164885582]
  Time-series representation learning can extract representations from data with temporal dynamics and sparse labels. We propose a new time-series representation learning method by combining the advantages of self-supervised tasks. We evaluate the proposed framework on three downstream tasks: time-series classification, forecasting, and anomaly detection.
  arXiv  Detail & Related papers  (2023-03-02T07:44:06Z)
• Benchmarking Node Outlier Detection on Graphs [90.29966986023403]
  Graph outlier detection is an emerging but crucial machine learning task with numerous applications. We present the first comprehensive unsupervised node outlier detection benchmark for graphs called UNOD.
  arXiv  Detail & Related papers  (2022-06-21T01:46:38Z)
• Enhancing Counterfactual Classification via Self-Training [9.484178349784264]
  We propose a self-training algorithm which imputes outcomes with categorical values for finite unseen actions in observational data to simulate a randomized trial through pseudolabeling. We demonstrate the effectiveness of the proposed algorithms on both synthetic and real datasets.
  arXiv  Detail & Related papers  (2021-12-08T18:42:58Z)
• Effective Evaluation of Deep Active Learning on Image Classification Tasks [10.27095298129151]
  We present a unified re-implementation of state-of-the-art active learning algorithms in the context of image classification. On the positive side, we show that AL techniques are 2x to 4x more label-efficient compared to RS with the use of data augmentation.
  arXiv  Detail & Related papers  (2021-06-16T23:29:39Z)
• Scalable Personalised Item Ranking through Parametric Density Estimation [53.44830012414444]
  Learning from implicit feedback is challenging because of the difficult nature of the one-class problem. Most conventional methods use a pairwise ranking approach and negative samplers to cope with the one-class problem. We propose a learning-to-rank approach, which achieves convergence speed comparable to the pointwise counterpart.
  arXiv  Detail & Related papers  (2021-05-11T03:38:16Z)
• Can Active Learning Preemptively Mitigate Fairness Issues? [66.84854430781097]
  dataset bias is one of the prevailing causes of unfairness in machine learning. We study whether models trained with uncertainty-based ALs are fairer in their decisions with respect to a protected class. We also explore the interaction of algorithmic fairness methods such as gradient reversal (GRAD) and BALD.
  arXiv  Detail & Related papers  (2021-04-14T14:20:22Z)
• DEALIO: Data-Efficient Adversarial Learning for Imitation from Observation [57.358212277226315]
  In imitation learning from observation IfO, a learning agent seeks to imitate a demonstrating agent using only observations of the demonstrated behavior without access to the control signals generated by the demonstrator. Recent methods based on adversarial imitation learning have led to state-of-the-art performance on IfO problems, but they typically suffer from high sample complexity due to a reliance on data-inefficient, model-free reinforcement learning algorithms. This issue makes them impractical to deploy in real-world settings, where gathering samples can incur high costs in terms of time, energy, and risk. We propose a more data-efficient IfO algorithm
  arXiv  Detail & Related papers  (2021-03-31T23:46:32Z)
• Modeling Score Distributions and Continuous Covariates: A Bayesian Approach [8.772459063453285]
  We develop a generative model of the match and non-match score distributions over continuous covariates. We use mixture models to capture arbitrary distributions and local basis functions. Three experiments demonstrate the accuracy and effectiveness of our approach.
  arXiv  Detail & Related papers  (2020-09-21T02:41:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.