Conf-GNNRec: Quantifying and Calibrating the Prediction Confidence for GNN-based Recommendation Methods

• URL: http://arxiv.org/abs/2505.16466v1
• Date: Thu, 22 May 2025 09:48:17 GMT
• Title: Conf-GNNRec: Quantifying and Calibrating the Prediction Confidence for GNN-based Recommendation Methods
• Authors: Meng Yan, Cai Xu, Xujing Wang, Ziyu Guan, Wei Zhao, Yuhang Zhou,
• Abstract summary: We propose a new method to quantify and calibrate the prediction confidence of GNN-based recommendations (Conf-GNNRec)<n>Specifically, we propose a rating calibration method that dynamically adjusts excessive ratings to mitigate overconfidence based on user personalization.<n>We also design a confidence loss function to reduce the overconfidence of negative samples and effectively improve recommendation performance.
• Score: 16.528524630468773
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recommender systems based on graph neural networks perform well in tasks such as rating and ranking. However, in real-world recommendation scenarios, noise such as user misuse and malicious advertisement gradually accumulates through the message propagation mechanism. Even if existing studies mitigate their effects by reducing the noise propagation weights, the severe sparsity of the recommender system still leads to the low-weighted noisy neighbors being mistaken as meaningful information, and the prediction result obtained based on the polluted nodes is not entirely trustworthy. Therefore, it is crucial to measure the confidence of the prediction results in this highly noisy framework. Furthermore, our evaluation of the existing representative GNN-based recommendation shows that it suffers from overconfidence. Based on the above considerations, we propose a new method to quantify and calibrate the prediction confidence of GNN-based recommendations (Conf-GNNRec). Specifically, we propose a rating calibration method that dynamically adjusts excessive ratings to mitigate overconfidence based on user personalization. We also design a confidence loss function to reduce the overconfidence of negative samples and effectively improve recommendation performance. Experiments on public datasets demonstrate the validity of Conf-GNNRec in prediction confidence and recommendation performance.

Related papers

• Uncertainty-Aware Graph Neural Networks: A Multi-Hop Evidence Fusion Approach [55.43914153271912]
  Graph neural networks (GNNs) excel in graph representation learning by integrating graph structure and node features.<n>Existing GNNs fail to account for the uncertainty of class probabilities that vary with the depth of the model, leading to unreliable and risky predictions in real-world scenarios.<n>We propose a novel Evidence Fusing Graph Neural Network (EFGNN for short) to achieve trustworthy prediction, enhance node classification accuracy, and make explicit the risk of wrong predictions.
  arXiv  Detail & Related papers  (2025-06-16T03:59:38Z)
• Enhance GNNs with Reliable Confidence Estimation via Adversarial Calibration Learning [30.450482094196243]
  Despite their impressive predictive performance, GNNs often exhibit poor confidence calibration.<n>This issue raises concerns about their reliability in high-stakes domains such as fraud detection, and risk assessment.<n>We propose a novel AdvCali framework that adaptively enhances calibration across different node groups.
  arXiv  Detail & Related papers  (2025-03-23T23:04:41Z)
• Quantifying calibration error in modern neural networks through evidence based theory [0.0]
  This paper introduces a novel framework for quantifying the trustworthiness of neural networks by incorporating subjective logic into the evaluation of Expected Error (ECE) We demonstrate the effectiveness of this approach through experiments on MNIST and CIFAR-10 datasets where post-calibration results indicate improved trustworthiness. The proposed framework offers a more interpretable and nuanced assessment of AI models, with potential applications in sensitive domains such as healthcare and autonomous systems.
  arXiv  Detail & Related papers  (2024-10-31T23:54:21Z)
• Revisiting Confidence Estimation: Towards Reliable Failure Prediction [53.79160907725975]
  We find a general, widely existing but actually-neglected phenomenon that most confidence estimation methods are harmful for detecting misclassification errors. We propose to enlarge the confidence gap by finding flat minima, which yields state-of-the-art failure prediction performance.
  arXiv  Detail & Related papers  (2024-03-05T11:44:14Z)
• A Trustworthiness Score to Evaluate DNN Predictions [1.5484595752241122]
  It is critical for safety during operation to know when deep neural networks' predictions are trustworthy or suspicious. We introduce the trustworthiness score (TS), a metric that provides a more transparent and effective way of providing confidence in predictions. We conduct a case study using YOLOv5 on persons detection to demonstrate our method and usage of TS and SS.
  arXiv  Detail & Related papers  (2023-01-21T00:48:18Z)
• Calibrating Deep Neural Networks using Explicit Regularisation and Dynamic Data Pruning [25.982037837953268]
  Deep neural networks (DNN) are prone to miscalibrated predictions, often exhibiting a mismatch between the predicted output and the associated confidence scores. We propose a novel regularization technique that can be used with classification losses, leading to state-of-the-art calibrated predictions at test time.
  arXiv  Detail & Related papers  (2022-12-20T05:34:58Z)
• Restricted Bernoulli Matrix Factorization: Balancing the trade-off between prediction accuracy and coverage in classification based collaborative filtering [45.335821132209766]
  We propose Restricted Bernoulli Matrix Factorization (ResBeMF) to enhance the performance of classification-based collaborative filtering. The proposed model provides a good balance in terms of the quality measures used compared to other recommendation models.
  arXiv  Detail & Related papers  (2022-10-05T13:48:19Z)
• TrustGNN: Graph Neural Network based Trust Evaluation via Learnable Propagative and Composable Nature [63.78619502896071]
  Trust evaluation is critical for many applications such as cyber security, social communication and recommender systems. We propose a new GNN based trust evaluation method named TrustGNN, which integrates smartly the propagative and composable nature of trust graphs. Specifically, TrustGNN designs specific propagative patterns for different propagative processes of trust, and distinguishes the contribution of different propagative processes to create new trust.
  arXiv  Detail & Related papers  (2022-05-25T13:57:03Z)
• Sampling-free Variational Inference for Neural Networks with Multiplicative Activation Noise [51.080620762639434]
  We propose a more efficient parameterization of the posterior approximation for sampling-free variational inference. Our approach yields competitive results for standard regression problems and scales well to large-scale image classification tasks.
  arXiv  Detail & Related papers  (2021-03-15T16:16:18Z)
• PAC Confidence Predictions for Deep Neural Network Classifiers [28.61937254015157]
  Key challenge for deploying deep neural networks (DNNs) in safety critical settings is the need to provide rigorous ways to quantify their uncertainty. We propose an algorithm for constructing predicted classification confidences for DNNs that comes with provable correctness guarantees.
  arXiv  Detail & Related papers  (2020-11-02T04:09:17Z)
• Unlabelled Data Improves Bayesian Uncertainty Calibration under Covariate Shift [100.52588638477862]
  We develop an approximate Bayesian inference scheme based on posterior regularisation. We demonstrate the utility of our method in the context of transferring prognostic models of prostate cancer across globally diverse populations.
  arXiv  Detail & Related papers  (2020-06-26T13:50:19Z)
• Binary Classification from Positive Data with Skewed Confidence [85.18941440826309]
  Positive-confidence (Pconf) classification is a promising weakly-supervised learning method. In practice, the confidence may be skewed by bias arising in an annotation process. We introduce the parameterized model of the skewed confidence, and propose the method for selecting the hyper parameter.
  arXiv  Detail & Related papers  (2020-01-29T00:04:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.