OT Score: An OT based Confidence Score for Unsupervised Domain Adaptation

• URL: http://arxiv.org/abs/2505.11669v1
• Date: Fri, 16 May 2025 20:09:05 GMT
• Title: OT Score: An OT based Confidence Score for Unsupervised Domain Adaptation
• Authors: Yiming Zhang, Sitong Liu, Alex Cloninger,
• Abstract summary: We introduce the Optimal Transport (OT) score, a confidence metric derived from a novel theoretical analysis.<n>We show that OT score significantly outperforms existing confidence metrics across diverse adaptation scenarios.
• Score: 4.544755512744677
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We address the computational and theoretical limitations of existing distributional alignment methods for unsupervised domain adaptation (UDA), particularly regarding the estimation of classification performance and confidence without target labels. Current theoretical frameworks for these methods often yield computationally intractable quantities and fail to adequately reflect the properties of the alignment algorithms employed. To overcome these challenges, we introduce the Optimal Transport (OT) score, a confidence metric derived from a novel theoretical analysis that exploits the flexibility of decision boundaries induced by Semi-Discrete Optimal Transport alignment. The proposed OT score is intuitively interpretable, theoretically rigorous, and computationally efficient. It provides principled uncertainty estimates for any given set of target pseudo-labels without requiring model retraining, and can flexibly adapt to varying degrees of available source information. Experimental results on standard UDA benchmarks demonstrate that classification accuracy consistently improves by identifying and removing low-confidence predictions, and that OT score significantly outperforms existing confidence metrics across diverse adaptation scenarios.

Related papers

• COIN: Uncertainty-Guarding Selective Question Answering for Foundation Models with Provable Risk Guarantees [51.5976496056012]
  COIN is an uncertainty-guarding selection framework that calibrates statistically valid thresholds to filter a single generated answer per question.<n>COIN estimates the empirical error rate on a calibration set and applies confidence interval methods to establish a high-probability upper bound on the true error rate.<n>We demonstrate COIN's robustness in risk control, strong test-time power in retaining admissible answers, and predictive efficiency under limited calibration data.
  arXiv  Detail & Related papers  (2025-06-25T07:04:49Z)
• TRUST: Test-time Resource Utilization for Superior Trustworthiness [15.031121920821109]
  We propose a novel test-time optimization method that accounts for the impact of such noise to produce more reliable confidence estimates.<n>This score defines a monotonic subset-selection function, where population accuracy consistently increases as samples with lower scores are removed.
  arXiv  Detail & Related papers  (2025-06-06T12:52:32Z)
• CLUE: Neural Networks Calibration via Learning Uncertainty-Error alignment [7.702016079410588]
  We introduce CLUE (Calibration via Learning Uncertainty-Error Alignment), a novel approach that aligns predicted uncertainty with observed error during training.<n>We show that CLUE achieves superior calibration quality and competitive predictive performance with respect to state-of-the-art approaches.
  arXiv  Detail & Related papers  (2025-05-28T19:23:47Z)
• Rectifying Conformity Scores for Better Conditional Coverage [75.73184036344908]
  We present a new method for generating confidence sets within the split conformal prediction framework.<n>Our method performs a trainable transformation of any given conformity score to improve conditional coverage while ensuring exact marginal coverage.
  arXiv  Detail & Related papers  (2025-02-22T19:54:14Z)
• Enhancing Trustworthiness of Graph Neural Networks with Rank-Based Conformal Training [17.120502204791407]
  Conformal Prediction can produce statistically guaranteed uncertainty estimates.<n>We propose a Rank-based CP during training framework to GNNs (RCP-GNN) for reliable uncertainty estimates.
  arXiv  Detail & Related papers  (2025-01-06T05:19:24Z)
• Beyond Conformal Predictors: Adaptive Conformal Inference with Confidence Predictors [0.0]
  Conformal prediction requires exchangeable data to ensure valid prediction sets at a user-specified significance level. Adaptive conformal inference (ACI) was introduced to address this limitation. We show that ACI does not require the use of conformal predictors; instead, it can be implemented with the more general confidence predictors.
  arXiv  Detail & Related papers  (2024-09-23T21:02:33Z)
• Optimal Baseline Corrections for Off-Policy Contextual Bandits [61.740094604552475]
  We aim to learn decision policies that optimize an unbiased offline estimate of an online reward metric. We propose a single framework built on their equivalence in learning scenarios. Our framework enables us to characterize the variance-optimal unbiased estimator and provide a closed-form solution for it.
  arXiv  Detail & Related papers  (2024-05-09T12:52:22Z)
• Cal-SFDA: Source-Free Domain-adaptive Semantic Segmentation with Differentiable Expected Calibration Error [50.86671887712424]
  The prevalence of domain adaptive semantic segmentation has prompted concerns regarding source domain data leakage. To circumvent the requirement for source data, source-free domain adaptation has emerged as a viable solution. We propose a novel calibration-guided source-free domain adaptive semantic segmentation framework.
  arXiv  Detail & Related papers  (2023-08-06T03:28:34Z)
• Learning to Estimate Without Bias [57.82628598276623]
  Gauss theorem states that the weighted least squares estimator is a linear minimum variance unbiased estimation (MVUE) in linear models. In this paper, we take a first step towards extending this result to non linear settings via deep learning with bias constraints. A second motivation to BCE is in applications where multiple estimates of the same unknown are averaged for improved performance.
  arXiv  Detail & Related papers  (2021-10-24T10:23:51Z)
• The Benefit of the Doubt: Uncertainty Aware Sensing for Edge Computing Platforms [10.86298377998459]
  We propose an efficient framework for predictive uncertainty estimation in NNs deployed on embedded edge systems. The framework is built from the ground up to provide predictive uncertainty based only on one forward pass. Our approach not only obtains robust and accurate uncertainty estimations but also outperforms state-of-the-art methods in terms of systems performance.
  arXiv  Detail & Related papers  (2021-02-11T11:44:32Z)
• SENTRY: Selective Entropy Optimization via Committee Consistency for Unsupervised Domain Adaptation [14.086066389856173]
  We propose a UDA algorithm that judges the reliability of a target instance based on its predictive consistency under a committee of random image transformations. Our algorithm then selectively minimizes predictive entropy to increase confidence on highly consistent target instances, while maximizing predictive entropy to reduce confidence on highly inconsistent ones. In combination with pseudo-label based approximate target class balancing, our approach leads to significant improvements over the state-of-the-art on 27/31 domain shifts from standard UDA benchmarks as well as benchmarks designed to stress-test adaptation under label distribution shift.
  arXiv  Detail & Related papers  (2020-12-21T16:24:50Z)
• Learning Calibrated Uncertainties for Domain Shift: A Distributionally Robust Learning Approach [150.8920602230832]
  We propose a framework for learning calibrated uncertainties under domain shifts. In particular, the density ratio estimation reflects the closeness of a target (test) sample to the source (training) distribution. We show that our proposed method generates calibrated uncertainties that benefit downstream tasks.
  arXiv  Detail & Related papers  (2020-10-08T02:10:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.