Conformal Fusion Under Missing Modalities

• URL: http://arxiv.org/abs/2608.07183v1
• Date: Fri, 07 Aug 2026 12:57:16 GMT
• Title: Conformal Fusion Under Missing Modalities
• Abstract summary: We argue that missing-modality robustness and calibrated uncertainty are a single coupled property, and introduce Modality-Conditioned Conformal Fusion (MCCF)<n>MCCF combines a multimodal bottleneck fusion backbone trained with modality dropout, per-modality evidential heads producing modality-decomposed Dirichlet distributions, and a Dempster-Shafer combination rule.<n>A Mondrian conformal calibration module keyed on the modality-presence mask then provides finite-sample group-conditional coverage for every non-empty modality subset.
• Score: 6.599344783327054
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal fusion architectures typically assume all modalities are available at inference, yet sensor failures, acquisition variability, and cost constraints routinely produce incomplete observations. Existing work treats modality absence as a prediction-accuracy problem, leaving a more basic question unanswered: whether a model's confidence estimates remain calibrated when an entire input stream is removed. We argue that missing-modality robustness and calibrated uncertainty are a single coupled property, and introduce Modality-Conditioned Conformal Fusion (MCCF), an architecture that addresses both at once. MCCF combines a multimodal bottleneck fusion backbone trained with modality dropout, per-modality evidential heads producing modality-decomposed Dirichlet distributions, and a Dempster-Shafer combination rule that fuses the per-modality evidence into a joint predictive distribution; an absent modality contributes vacuous evidence that is structurally ignored, so the fused uncertainty automatically reflects the reduced information without test-time imputation. A Mondrian conformal calibration module keyed on the modality-presence mask then provides finite-sample group-conditional coverage for every non-empty modality subset. MCCF is, to our knowledge, the first method with formal coverage guarantees under arbitrary modality availability through architectural integration rather than post-hoc recalibration, and the evidential decomposition yields per-modality vacuity scores that localise uncertainty to the absent modality responsible. Across a synthetic problem and three real multimodal benchmarks, MCCF holds its target coverage on every modality-presence subset, substantially narrows the coverage gap between full and partial modalities relative to a marginal split-conformal baseline, and imposes no measurable accuracy cost relative to temperature-scaled and evidential baselines.

Related papers

• Rethinking Modality Reliability in Multimodal Sentiment Analysis with Incomplete Observations [9.993152436695421]
  We argue that modality reliability is a central variable in incomplete-observation settings. Failure to model it explicitly gives rise to two related issues.<n>We propose Modality Reliability-Calibrated Framework for Multimodal Sentiment Analysis with incomplete observations.<n> MRCF contains a Reliability-Aware Branch that estimates sample-specific modality reliability from intramodal quality cues and cross-modal semantic consistency.<n>Experiments on CMU-MOSI, CMU-MOSEI, and CH-SIMS show that MRCF achieves strong performance under standard incomplete-observation protocols.
  arXiv  Detail & Related papers  (2026-08-04T13:02:48Z)
• Adaptive Modality Reliability Diagnosis and Restoration for Robust Multimodal Intent Recognition [2.2012643583422347]
  Multimodal intent recognition combines linguistic, acoustic, and visual evidence, but individual modalities may be noisy, missing, semantically conflicting, or disproportionately dominant.<n>We propose PRIME, a closed-loop reliability guided framework that jointly diagnoses, restores, and reassesses modality quality at the sample level.<n> Experiments on multimodal intent-recognition benchmarks show that PRIME maintains competitive clean-data performance while improving under missing, noisy, conflicting, and modality-imbalanced conditions.
  arXiv  Detail & Related papers  (2026-08-04T11:14:13Z)
• Dual-Edged Homogeneous-Modality Similarity: Towards Visible-Infrared Modality-Incomplete Person Re-Identification with Modality Adaptive Matching [60.86381869046298]
  We formalize the Visible-Infrared Modality-Incomplete Re-Identification (VIMI-ReID) task.<n>The unpredictable modality combinations and inherent similarity of homogeneous-modality samples in VIMI-ReID cause a significant performance drop.<n>We propose the Modality Adaptive Matching Transformer (MAMT) to extract modality-specific and modality-shared features.
  arXiv  Detail & Related papers  (2026-07-21T04:10:20Z)
• Set-Inclusive Uncertainty Modeling for Robust Brain Tumor Segmentation [17.88630413987641]
  We propose a probabilistic representation framework that models representations as Gaussian distributions.<n>We regularize the mean from each partial configuration toward its full-modality counterpart, while scaling the variance with the discrepancy between their aligned means.<n>Experiments on BraTS 2018 and 2020 demonstrate that our approach offers superior performance over baselines across diverse missing-modality scenarios.
  arXiv  Detail & Related papers  (2026-06-29T14:35:55Z)
• PRA-PoE: Robust Multimodal Alzheimer's Diagnosis with Arbitrary Missing Modalities [9.014577470296013]
  Missing modalities are prevalent in real-world Alzheimer's disease (AD) assessment.<n>Existing approaches that rely on implicit imputation or modality synthesis often fail to explicitly model modality availability and uncertainty.<n>We propose Prototype-anchored Representation Alignment (PRA) and an Uncertainty-aware Product of Experts (UA-PoE) fusion mechanism.
  arXiv  Detail & Related papers  (2026-05-13T06:54:21Z)
• Multimodal Learning on Low-Quality Data with Conformal Predictive Self-Calibration [72.0672328514289]
  Multimodal learning often grapples with the challenge of low-quality data.<n>We propose a unified framework, termed Conformal Predictive Self-Calibration.<n>Our framework consistently outperforms existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-05-05T14:48:52Z)
• Towards Trustworthy Depression Estimation via Disentangled Evidential Learning [50.22167852149165]
  We propose EviDep, an evidential learning framework that jointly quantifies depression severity.<n>EviDep enforces strict information integrity to guarantee robust evidence synthesis.<n>It achieves state-of-the-art predictive accuracy and superior uncertainty calibration, delivering a robust fail-safe mechanism for trustworthy clinical screening.
  arXiv  Detail & Related papers  (2026-04-17T13:27:11Z)
• Calibrated Multimodal Representation Learning with Missing Modalities [100.55774771852468]
  Multimodal representation learning harmonizes distinct modalities by aligning them into a unified latent space.<n>Recent research generalizes traditional cross-modal alignment to produce enhanced multimodal synergy but requires all modalities to be present for a common instance.<n>We provide theoretical insights into this issue from an anchor shift perspective.<n>We propose CalMRL for multimodal representation learning to calibrate incomplete alignments caused by missing modalities.
  arXiv  Detail & Related papers  (2025-11-15T05:01:43Z)
• UniMRSeg: Unified Modality-Relax Segmentation via Hierarchical Self-Supervised Compensation [104.59740403500132]
  Multi-modal image segmentation faces real-world deployment challenges from incomplete/corrupted modalities degrading performance.<n>We propose a unified modality-relax segmentation network (UniMRSeg) through hierarchical self-supervised compensation (HSSC)<n>Our approach hierarchically bridges representation gaps between complete and incomplete modalities across input, feature and output levels.
  arXiv  Detail & Related papers  (2025-09-19T17:29:25Z)
• Imputation-Powered Inference [2.676349883103404]
  imputation-powered inference (IPI) is a model-lean framework that combines the flexibility of blackbox imputation with bias correction using fully observed data.<n>IPI may substantially improve subpopulation efficiency relative to complete-case analysis, while maintaining statistical validity in settings where both doubly robust estimators and naive imputation fail to achieve nominal coverage.
  arXiv  Detail & Related papers  (2025-09-17T07:48:54Z)
• Benchmarking Multi-modal Semantic Segmentation under Sensor Failures: Missing and Noisy Modality Robustness [61.87055159919641]
  Multi-modal semantic segmentation (MMSS) addresses the limitations of single-modality data by integrating complementary information across modalities.<n>Despite notable progress, a significant gap persists between research and real-world deployment due to variability and uncertainty in multi-modal data quality.<n>We introduce a robustness benchmark that evaluates MMSS models under three scenarios: Entire-Missing Modality (EMM), Random-Missing Modality (RMM), and Noisy Modality (NM)
  arXiv  Detail & Related papers  (2025-03-24T08:46:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.