Dynamic Distribution-Aware Uncertainty Tracking in Vision-Language Representation Learning

• URL: http://arxiv.org/abs/2608.09011v2
• Date: Wed, 19 Aug 2026 06:36:51 GMT
• Title: Dynamic Distribution-Aware Uncertainty Tracking in Vision-Language Representation Learning
• Abstract summary: Uncertainty Quantification aims to measure the reliability of model predictions.<n>Post-hoc approaches are widely adopted due to their lightweight nature.<n>We propose a Dynamic Distribution-Aware Uncertainty Quantification framework (DDA-UQ) that shifts the paradigm from static mapping to a dynamic distribution-aware process.
• Score: 13.524679557181997
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Uncertainty Quantification (UQ) aims to measure the reliability of model predictions, serving as a critical safeguard for deploying Vision-Language Models (VLMs) in safety-critical scenarios. Post-hoc approaches are widely adopted due to their lightweight nature, mapping the outputs of VLMs to uncertainty measures through learnable modules or inductive summarization. However, Post-hoc approaches remain inherently confined to fitting the failure patterns of the source domain, ignoring the dynamic nature of test distributions. To address this challenge, we propose a Dynamic Distribution-Aware Uncertainty Quantification framework (DDA-UQ) that shifts the paradigm from static mapping to a dynamic distribution-aware process. During training, we leverage a Gaussian Mixture Model to model the VVLMs'embedding space and extract distributional evidence, thereby dynamically deriving uncertainty estimates. During inference, the design dynamically responds to changes in the data distribution. Extensive experiments demonstrate that our approach significantly outperforms state-of-the-art methods.

Related papers

• Estimating Semantic Ambiguity via Gaussian Context Distributions for VLM-Driven Traversability Analysis [14.31446990918297]
  We present a novel pipeline for vision-based traversability estimation that explicitly models contextual uncertainty.<n>Our approach utilizes Conceptual Anchoring to ground open-vocabulary VLM predictions onto a continuous physical traversability scale.<n>We derive both a dense traversability map and a dense uncertainty map based on the statistical properties of the distribution.
  arXiv  Detail & Related papers  (2026-09-08T11:18:32Z)
• DUD: Decoupled Update Dynamics for Reliable Uncertainty Quantification in Large Language Models [16.097057987606714]
  We introduce textbfDecoupled textbfUpdate textbfDynamics textbf(DUD), a framework that explicitly decouples FFN and Attention contributions via noise-induced causal interventions.<n>By quantifying the independent restoration capabilities of each module, we construct a dual-stream dynamic profile that captures the model's internal fragility.
  arXiv  Detail & Related papers  (2026-08-04T10:04:33Z)
• Perturbation-Based Uncertainty for Failure Detection in Vision-Language-Action Models [5.736588561666141]
  We propose a model-agnostic framework for inference-time uncertainty estimation through hidden activation perturbations.<n>We show that perturbation-based uncertainty consistently improves failure detection under distribution shift compared to sampling-based uncertainty.
  arXiv  Detail & Related papers  (2026-06-18T05:41:45Z)
• FUSE: Quantifying Uncertainty in Vision-Language Models by Bayesian Fusing Epistemic and Aleatoric Uncertainty [20.637224080557534]
  In many applications, such as robotics, it is crucial to quantify the uncertainty in the output of vision-language models.<n>We develop FUSE, a probabilistic framework for capturing two complementary sources of uncertainty in vision-language modeling.
  arXiv  Detail & Related papers  (2026-06-01T22:11:00Z)
• Quantifying Aleatoric Uncertainty of In-Context Learning for Robust Measure of LLM Prediction Confidence [6.28866496877782]
  In-Context Learning (ICL) allows LLMs to adapt to new tasks from a few demonstrations, but its reliability remains a concern.<n>We introduce a concept of self-function vectors, built upon Bayesian views and the mechanistic interpretability of ICL.<n>These vectors leverage internal model representations to model the latent concept learned during in-context prompting.<n>We show that our proposed methodology can measure uncertainty of LLM predictions more reliably than existing alternative methods.
  arXiv  Detail & Related papers  (2026-04-28T09:47:40Z)
• Reshaping Action Error Distributions for Reliable Vision-Language-Action Models [69.38615670891038]
  In robotic manipulation, vision-language-action (VLA) models have emerged as a promising paradigm for learning generalizable and scalable robot policies.<n>We focus on continuous-action VLA models and move beyond conventional MSE-based regression by reshaping action error distributions during training.<n>We evaluate our approaches across standard, few-shot, and noisy settings on multiple representative VLA architectures.
  arXiv  Detail & Related papers  (2026-02-04T05:37:09Z)
• From Passive Metric to Active Signal: The Evolving Role of Uncertainty Quantification in Large Language Models [77.04403907729738]
  This survey charts the evolution of uncertainty from a passive diagnostic metric to an active control signal guiding real-time model behavior.<n>We demonstrate how uncertainty is leveraged as an active control signal across three frontiers.<n>This survey argues that mastering the new trend of uncertainty is essential for building the next generation of scalable, reliable, and trustworthy AI.
  arXiv  Detail & Related papers  (2026-01-22T06:21:31Z)
• Model-Based Diffusion Sampling for Predictive Control in Offline Decision Making [48.998030470623384]
  offline decision-making requires reliable behaviors from fixed datasets without further interaction.<n>We propose a compositional model-based diffusion framework consisting of: (i) a planner that generates diverse, task-aligned trajectories; (ii) a dynamics model that enforces consistency with the underlying system dynamics; and (iii) a ranker module that selects behaviors aligned with the task objectives.
  arXiv  Detail & Related papers  (2025-12-09T06:26:02Z)
• ESI: Epistemic Uncertainty Quantification via Semantic-preserving Intervention for Large Language Models [23.44710972442814]
  Uncertainty Quantification (UQ) is a promising approach to improve model reliability, yet the uncertainty of Large Language Models (LLMs) is non-trivial.<n>We propose a novel grey-box uncertainty quantification method that measures the variation in model outputs before and after semantic-preserving intervention.
  arXiv  Detail & Related papers  (2025-10-15T02:46:43Z)
• Advancing Reliable Test-Time Adaptation of Vision-Language Models under Visual Variations [67.35596444651037]
  Vision-language models (VLMs) exhibit remarkable zero-shot capabilities but struggle with distribution shifts in downstream tasks when labeled data is unavailable.<n>We propose a Reliable Test-time Adaptation (ReTA) method that enhances reliability from two perspectives.
  arXiv  Detail & Related papers  (2025-07-13T05:37:33Z)
• Consistent World Models via Foresight Diffusion [56.45012929930605]
  We argue that a key bottleneck in learning consistent diffusion-based world models lies in the suboptimal predictive ability.<n>We propose Foresight Diffusion (ForeDiff), a diffusion-based world modeling framework that enhances consistency by decoupling condition understanding from target denoising.
  arXiv  Detail & Related papers  (2025-05-22T10:01:59Z)
• HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Model [54.64088247291416]
  A fundamental objective of manipulation policy design is to endow robots to comprehend human instructions, reason about scene cues, and execute generalized actions in dynamic environments.<n>Recent autoregressive vision-language-action (VLA) methods inherit common-sense reasoning capabilities from vision-language models (VLMs) for next action-token prediction.<n>We introduce HybridVLA, a unified framework that absorbs the continuous nature of diffusion-based actions and the contextual reasoning of autoregression.
  arXiv  Detail & Related papers  (2025-03-13T17:59:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.