Focus on the Likely: Test-time Instance-based Uncertainty Removal

• URL: http://arxiv.org/abs/2505.03819v2
• Date: Fri, 16 May 2025 15:21:29 GMT
• Title: Focus on the Likely: Test-time Instance-based Uncertainty Removal
• Authors: Johannes Schneider,
• Abstract summary: We propose two novel test-time fine-tuning methods to improve uncertain model predictions.<n>Instead of greedily selecting the most likely class, we introduce an additional step, emphfocus on the likely classes, to refine predictions.
• Score: 1.8592384822257952
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We ask: Does focusing on classes predicted as likely improve model predictions? We aim for an affirmative answer by proposing two novel test-time fine-tuning methods to improve uncertain model predictions. Instead of greedily selecting the most likely class, we introduce an additional step, \emph{focus on the likely classes}, to refine predictions. By applying a theoretically motivated single gradient descent step with a large learning rate, we refine predictions when an initial forward pass indicates high uncertainty. This aligns predictions more closely with the ideal of assigning zero probability to less plausible outcomes. The experimental evaluation demonstrates accuracy gains for one of our methods, which emphasizes shared features among likely classes, across diverse text and image domain models. %Our theoretical discussion provides a deeper understanding, highlighting the varying impact of shared and non-shared features among (focus) classes. %Our discussion also suggests an interesting view on standard, offline training vs. test-time training: Opposing optimization rationales regarding breadth of feature dependence are preferable during each training phase.

Related papers

• Bayesian Test-Time Adaptation for Vision-Language Models [51.93247610195295]
  Test-time adaptation with pre-trained vision-language models, such as CLIP, aims to adapt the model to new, potentially out-of-distribution test data.<n>We propose a novel approach, textbfBayesian textbfClass textbfAdaptation (BCA), which in addition to continuously updating class embeddings to adapt likelihood, also uses the posterior of incoming samples to continuously update the prior for each class embedding.
  arXiv  Detail & Related papers  (2025-03-12T10:42:11Z)
• Conformalized Selective Regression [2.3964255330849356]
  We propose a novel approach to selective regression by leveraging conformal prediction. We show how our proposed approach, conformalized selective regression, demonstrates an advantage over multiple state-of-the-art baselines.
  arXiv  Detail & Related papers  (2024-02-26T04:43:50Z)
• ASPEST: Bridging the Gap Between Active Learning and Selective Prediction [56.001808843574395]
  Selective prediction aims to learn a reliable model that abstains from making predictions when uncertain. Active learning aims to lower the overall labeling effort, and hence human dependence, by querying the most informative examples. In this work, we introduce a new learning paradigm, active selective prediction, which aims to query more informative samples from the shifted target domain.
  arXiv  Detail & Related papers  (2023-04-07T23:51:07Z)
• Calibrated Selective Classification [34.08454890436067]
  We develop a new approach to selective classification in which we propose a method for rejecting examples with "uncertain" uncertainties. We present a framework for learning selectively calibrated models, where a separate selector network is trained to improve the selective calibration error of a given base model. We demonstrate the empirical effectiveness of our approach on multiple image classification and lung cancer risk assessment tasks.
  arXiv  Detail & Related papers  (2022-08-25T13:31:09Z)
• Selective Prediction via Training Dynamics [31.708701583736644]
  We show that state-of-the-art selective prediction performance can be attained solely from studying the training dynamics of a model.<n>In particular, we reject data points exhibiting too much disagreement with the final prediction at late stages in training.<n>The proposed rejection mechanism is domain-agnostic (i.e., it works for both discrete and real-valued prediction) and can be flexibly combined with existing selective prediction approaches.
  arXiv  Detail & Related papers  (2022-05-26T17:51:29Z)
• Uncertainty estimation of pedestrian future trajectory using Bayesian approximation [137.00426219455116]
  Under dynamic traffic scenarios, planning based on deterministic predictions is not trustworthy. The authors propose to quantify uncertainty during forecasting using approximation which deterministic approaches fail to capture. The effect of dropout weights and long-term prediction on future state uncertainty has been studied.
  arXiv  Detail & Related papers  (2022-05-04T04:23:38Z)
• Masked prediction tasks: a parameter identifiability view [49.533046139235466]
  We focus on the widely used self-supervised learning method of predicting masked tokens. We show that there is a rich landscape of possibilities, out of which some prediction tasks yield identifiability, while others do not.
  arXiv  Detail & Related papers  (2022-02-18T17:09:32Z)
• Taming Overconfident Prediction on Unlabeled Data from Hindsight [50.9088560433925]
  Minimizing prediction uncertainty on unlabeled data is a key factor to achieve good performance in semi-supervised learning. This paper proposes a dual mechanism, named ADaptive Sharpening (ADS), which first applies a soft-threshold to adaptively mask out determinate and negligible predictions. ADS significantly improves the state-of-the-art SSL methods by making it a plug-in.
  arXiv  Detail & Related papers  (2021-12-15T15:17:02Z)
• HYPER: Learned Hybrid Trajectory Prediction via Factored Inference and Adaptive Sampling [27.194900145235007]
  We introduce HYPER, a general and expressive hybrid prediction framework. By modeling traffic agents as a hybrid discrete-continuous system, our approach is capable of predicting discrete intent changes over time. We train and validate our model on the Argoverse dataset, and demonstrate its effectiveness through comprehensive ablation studies and comparisons with state-of-the-art models.
  arXiv  Detail & Related papers  (2021-10-05T20:20:10Z)
• How to Evaluate Uncertainty Estimates in Machine Learning for Regression? [1.4610038284393165]
  We show that both approaches to evaluating the quality of uncertainty estimates have serious flaws. Firstly, both approaches cannot disentangle the separate components that jointly create the predictive uncertainty. Thirdly, the current approach to test prediction intervals directly has additional flaws.
  arXiv  Detail & Related papers  (2021-06-07T07:47:46Z)
• Predicting MOOCs Dropout Using Only Two Easily Obtainable Features from the First Week's Activities [56.1344233010643]
  Several features are considered to contribute towards learner attrition or lack of interest, which may lead to disengagement or total dropout. This study aims to predict dropout early-on, from the first week, by comparing several machine-learning approaches.
  arXiv  Detail & Related papers  (2020-08-12T10:44:49Z)
• Video Prediction via Example Guidance [156.08546987158616]
  In video prediction tasks, one major challenge is to capture the multi-modal nature of future contents and dynamics. In this work, we propose a simple yet effective framework that can efficiently predict plausible future states.
  arXiv  Detail & Related papers  (2020-07-03T14:57:24Z)
• Ambiguity in Sequential Data: Predicting Uncertain Futures with Recurrent Models [110.82452096672182]
  We propose an extension of the Multiple Hypothesis Prediction (MHP) model to handle ambiguous predictions with sequential data. We also introduce a novel metric for ambiguous problems, which is better suited to account for uncertainties.
  arXiv  Detail & Related papers  (2020-03-10T09:15:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.