HARMONY: Hidden Activation Representations and Model Output-Aware Uncertainty Estimation for Vision-Language Models

• URL: http://arxiv.org/abs/2510.22171v1
• Date: Sat, 25 Oct 2025 05:45:18 GMT
• Title: HARMONY: Hidden Activation Representations and Model Output-Aware Uncertainty Estimation for Vision-Language Models
• Authors: Erum Mushtaq, Zalan Fabian, Yavuz Faruk Bakman, Anil Ramakrishna, Mahdi Soltanolkotabi, Salman Avestimehr,
• Abstract summary: Uncertainty Estimation plays a central role in quantifying the reliability of model outputs.<n>Most existing probability-based UE approaches rely on output probability distributions aggregating token probabilities into a single uncertainty score.<n>We propose a novel UE framework, HARMONY, that jointly leverages fused multimodal information in model activations and the output distribution of the VLM.
• Score: 42.91752946934796
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The growing deployment of Vision-Language Models (VLMs) in high-stakes applications such as autonomous driving and assistive technologies for visually impaired individuals necessitates reliable mechanisms to assess the trustworthiness of their generation. Uncertainty Estimation (UE) plays a central role in quantifying the reliability of model outputs and reducing unsafe generations via selective prediction. In this regard, most existing probability-based UE approaches rely on output probability distributions, aggregating token probabilities into a single uncertainty score using predefined functions such as length-normalization. Another line of research leverages model hidden representations and trains MLP-based models to predict uncertainty. However, these methods often fail to capture the complex multimodal relationships between semantic and textual tokens and struggle to identify biased probabilities often influenced by language priors. Motivated by these observations, we propose a novel UE framework, HARMONY, that jointly leverages fused multimodal information in model activations and the output distribution of the VLM to determine the reliability of responses. The key hypothesis of our work is that both the model's internal belief in its visual understanding, captured by its hidden representations, and the produced token probabilities carry valuable reliability signals that can be jointly leveraged to improve UE performance, surpassing approaches that rely on only one of these components. Experimental results on three open-ended VQA benchmarks, A-OKVQA, VizWiz, and PathVQA, and three state-of-the-art VLMs, LLaVa-7b, LLaVA-13b and InstructBLIP demonstrate that our method consistently performs on par with or better than existing approaches, achieving up to 4\% improvement in AUROC, and 6\% in PRR, establishing new state of the art in uncertainty estimation for VLMs.

Related papers

• Leveraging LLM Parametric Knowledge for Fact Checking without Retrieval [60.25608870901428]
  Trustworthiness is a core research challenge for agentic AI systems built on Large Language Models (LLMs)<n>We propose the task of fact-checking without retrieval, focusing on the verification of arbitrary natural language claims, independent of their source robustness.
  arXiv  Detail & Related papers  (2026-03-05T18:42:51Z)
• VAUQ: Vision-Aware Uncertainty Quantification for LVLM Self-Evaluation [22.921677603408188]
  Large Vision-Language Models (LVLMs) frequently hallucinate, limiting their safe deployment in real-world applications.<n>We propose VAUQ, a vision-aware uncertainty quantification framework for LVLM self-evaluation.<n>VAUQ explicitly measures how strongly a model's output depends on visual evidence.
  arXiv  Detail & Related papers  (2026-02-24T16:11:14Z)
• Trust in One Round: Confidence Estimation for Large Language Models via Structural Signals [13.89434979851652]
  Large language models (LLMs) are increasingly deployed in domains where errors carry high social, scientific, or safety costs.<n>We present Structural Confidence, a single-pass, model-agnostic framework that enhances output correctness prediction.
  arXiv  Detail & Related papers  (2026-02-01T02:35:59Z)
• Explaining Machine Learning Predictive Models through Conditional Expectation Methods [0.0]
  MUCE is a model-agnostic method for local explainability designed to capture prediction changes from feature interactions.<n>Two quantitative indices, stability and uncertainty, summarize local behavior and assess model reliability.<n>Results show that MUCE effectively captures complex local model behavior, while the stability and uncertainty indices provide meaningful insight into prediction confidence.
  arXiv  Detail & Related papers  (2026-01-12T08:34:36Z)
• Mind the Generation Process: Fine-Grained Confidence Estimation During LLM Generation [63.49409574310576]
  Large language models (LLMs) exhibit overconfidence, assigning high confidence scores to incorrect predictions.<n>We introduce FineCE, a novel confidence estimation method that delivers accurate, fine-grained confidence scores during text generation.<n>Our code and all baselines used in the paper are available on GitHub.
  arXiv  Detail & Related papers  (2025-08-16T13:29:35Z)
• Can LLMs Detect Their Confabulations? Estimating Reliability in Uncertainty-Aware Language Models [24.72990207218907]
  Large Language Models (LLMs) are prone to generating fluent but incorrect content, known as confabulation.<n>We investigate how in-context information influences model behavior and whether LLMs can identify their unreliable responses.
  arXiv  Detail & Related papers  (2025-08-11T16:12:36Z)
• A Context-Aware Dual-Metric Framework for Confidence Estimation in Large Language Models [6.62851757612838]
  Current confidence estimation methods for large language models (LLMs) neglect the relevance between responses and contextual information.<n>We propose CRUX, which integrates context faithfulness and consistency for confidence estimation via two novel metrics.<n> Experiments across three benchmark datasets demonstrate CRUX's effectiveness, achieving the highest AUROC than existing baselines.
  arXiv  Detail & Related papers  (2025-08-01T12:58:34Z)
• Seeing is Believing, but How Much? A Comprehensive Analysis of Verbalized Calibration in Vision-Language Models [15.158475816860427]
  Uncertainty is essential for assessing the reliability and trustworthiness of modern AI systems.<n> verbalized uncertainty, where models express their confidence through natural language, has emerged as a lightweight and interpretable solution.<n>However, its effectiveness in vision-language models (VLMs) remains insufficiently studied.
  arXiv  Detail & Related papers  (2025-05-26T17:16:36Z)
• Uncertainty Quantification for LLMs through Minimum Bayes Risk: Bridging Confidence and Consistency [66.96286531087549]
  Uncertainty quantification (UQ) methods for Large Language Models (LLMs) encompass a variety of approaches.<n>We propose a novel approach to integrating model confidence with output consistency, resulting in a family of efficient and robust UQ methods.<n>We evaluate our approach across various tasks such as question answering, abstractive summarization, and machine translation.
  arXiv  Detail & Related papers  (2025-02-07T14:30:12Z)
• Label-Confidence-Aware Uncertainty Estimation in Natural Language Generation [8.635811152610604]
  Uncertainty Quantification (UQ) is crucial for ensuring the safety and robustness of AI systems.<n>We propose a label-confidence-aware (LCA) uncertainty estimation based on Kullback-Leibler divergence bridging between samples and label source.
  arXiv  Detail & Related papers  (2024-12-10T07:35:23Z)
• On the Fairness, Diversity and Reliability of Text-to-Image Generative Models [68.62012304574012]
  multimodal generative models have sparked critical discussions on their reliability, fairness and potential for misuse.<n>We propose an evaluation framework to assess model reliability by analyzing responses to global and local perturbations in the embedding space.<n>Our method lays the groundwork for detecting unreliable, bias-injected models and tracing the provenance of embedded biases.
  arXiv  Detail & Related papers  (2024-11-21T09:46:55Z)
• A Survey on Uncertainty Toolkits for Deep Learning [3.113304966059062]
  We present the first survey on toolkits for uncertainty estimation in deep learning (DL) We investigate 11 toolkits with respect to modeling and evaluation capabilities. While the first two provide a large degree of flexibility and seamless integration into their respective framework, the last one has the larger methodological scope.
  arXiv  Detail & Related papers  (2022-05-02T17:23:06Z)
• Trust but Verify: Assigning Prediction Credibility by Counterfactual Constrained Learning [123.3472310767721]
  Prediction credibility measures are fundamental in statistics and machine learning. These measures should account for the wide variety of models used in practice. The framework developed in this work expresses the credibility as a risk-fit trade-off.
  arXiv  Detail & Related papers  (2020-11-24T19:52:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.