Related papers: Unified Multimodal Uncertain Inference

Unified Multimodal Uncertain Inference

URL: http://arxiv.org/abs/2604.08701v2
Date: Mon, 13 Apr 2026 15:22:56 GMT
Title: Unified Multimodal Uncertain Inference
Abstract summary: Unified Multimodal Uncertain Inference (UMUI) is a task spanning text, audio, and video.<n>We introduce CLUE (Calibrated Latent Uncertainty Estimation), which combines self-consistent teacher calibration and distribution-based calibrated confidence probing to produce predictions.
Score: 81.64340342055327
License: http://creativecommons.org/licenses/by/4.0/
Abstract: We introduce Unified Multimodal Uncertain Inference (UMUI), a multimodal inference task spanning text, audio, and video, where models must produce calibrated probability estimates of hypotheses conditioned on a premise in any modality or combination. While uncertain inference has been explored in text, extension to other modalities has been limited to single-modality binary entailment judgments, leaving no framework for fine-grained probabilistic reasoning in or across other modalities. To address this, we curate a human-annotated evaluation set with scalar probability judgments across audio, visual, and audiovisual settings, and additionally evaluate on existing text and audio benchmarks. We introduce CLUE (Calibrated Latent Uncertainty Estimation), which combines self-consistent teacher calibration and distribution-based confidence probing to produce calibrated predictions. We demonstrate that our 3B-parameter model achieves equivalent or stronger performance than baselines up to 32B parameters across all modalities.

Related papers

Walking Through Uncertainty: An Empirical Study of Uncertainty Estimation for Audio-Aware Large Language Models [54.041320081289996]
We present the first systematic empirical study of uncertainty estimation in audio-aware large language models (ALLMs)<n>We benchmark five representative methods, including predictive entropy, length-normalized entropy, semantic entropy, discrete semantic entropy, and P(True)<n>Our results reveal two key findings. First, semantic-level and verification-based methods consistently outperform token-level baselines on general audio reasoning benchmarks.
arXiv Detail & Related papers (2026-04-28T12:56:22Z)
MMEB-V3: Measuring the Performance Gaps of Omni-Modality Embedding Models [70.34265674686516]
Multimodal embedding models aim to map heterogeneous inputs, such as text, images, videos, and audio, into a shared semantic space.<n>We introduce MMEB-V3, a benchmark that evaluates embeddings across text, image, video, audio, as well as agent-centric scenarios.<n>We conduct a systematic analysis of full-modality embeddings and identify three key findings.
arXiv Detail & Related papers (2026-04-25T14:15:05Z)
Omni-Modal Dissonance Benchmark: Systematically Breaking Modality Consensus to Probe Robustness and Calibrated Abstention [7.218987720005679]
We introduce OMD-Bench, where all modalities are initially congruent.<n>We then systematically corrupt to isolate each modality's contribution.<n>The benchmark comprises 4,080 instances spanning 27 anchors across eight corruption conditions.
arXiv Detail & Related papers (2026-03-28T08:29:15Z)
On Calibration of Large Language Models: From Response To Capability [66.59139960234326]
Large language models (LLMs) are widely deployed as general-purpose problem solvers.<n>We introduce capability calibration, which targets the model's expected accuracy on a query.<n>Our results demonstrate that capability-calibrated confidence improves pass@$k$ prediction and inference budget allocation.
arXiv Detail & Related papers (2026-02-14T01:07:45Z)
Heterogeneous Uncertainty-Guided Composed Image Retrieval with Fine-Grained Probabilistic Learning [49.28548464288051]
Composed Image Retrieval (CIR) enables image search by combining a reference image with modification text.<n>In intrinsic noise in CIR triplets incurs intrinsic uncertainty and threatens the model's robustness.<n>This paper introduces a Heterogeneous Uncertainty-Guided (HUG) paradigm to overcome these limitations.
arXiv Detail & Related papers (2026-01-16T16:05:49Z)
Confidence in Large Language Model Evaluation: A Bayesian Approach to Limited-Sample Challenges [13.526258635654882]
This study introduces a Bayesian approach for large language models (LLMs) capability assessment.<n>We treat model capabilities as latent variables and leverage a curated query set to induce discriminative responses.<n> Experimental evaluations with GPT-series models demonstrate that the proposed method achieves superior discrimination compared to conventional evaluation methods.
arXiv Detail & Related papers (2025-04-30T04:24:50Z)
Confidence Estimation via Sequential Likelihood Mixing [46.69347918899963]
We present a universal framework for constructing confidence sets based on sequential likelihood mixing.<n>We establish fundamental connections between sequential mixing, Bayesian inference and regret inequalities from online estimation.<n>We illustrate the power of the framework by deriving tighter confidence sequences for classical settings.
arXiv Detail & Related papers (2025-02-20T16:16:34Z)
Prediction-Powered E-Values [0.66567375919026]
We apply ideas of prediction-powered inference to e-values.<n>We show that every inference procedure that can be framed in terms of e-values has a prediction-powered counterpart.<n>Our approach is modular and easily integrable into existing algorithms.
arXiv Detail & Related papers (2025-02-06T18:36:01Z)
Trusted Multi-View Classification with Dynamic Evidential Fusion [73.35990456162745]
We propose a novel multi-view classification algorithm, termed trusted multi-view classification (TMC) TMC provides a new paradigm for multi-view learning by dynamically integrating different views at an evidence level. Both theoretical and experimental results validate the effectiveness of the proposed model in accuracy, robustness and trustworthiness.
arXiv Detail & Related papers (2022-04-25T03:48:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.