MuteBench: Modality Unavailability Tolerance Evaluation for Incomplete Multimodal Fusion

• URL: http://arxiv.org/abs/2605.15235v1
• Date: Wed, 13 May 2026 19:32:11 GMT
• Title: MuteBench: Modality Unavailability Tolerance Evaluation for Incomplete Multimodal Fusion
• Abstract summary: We present MuteBench, a benchmark covering 9 datasets from 7 clinical domains.<n>We find that architecture family is the strongest predictor of robustness, outweighing parameter count.<n>Channel-independent models tolerate modality missing well but can be sensitive to within-modality missing.
• Score: 37.86089523035199
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal physiological data powers clinical AI systems from intensive care units to wearable devices, but sensors routinely fail in practice. Two failure modes are common: modality missing, where an entire channel is absent, and within-modality missing, where a contiguous time segment is lost. No existing benchmark evaluates multiple fusion architectures under both failure modes at controlled severity levels across diverse clinical datasets. We present MuteBench, a benchmark covering 9 datasets from 7 clinical domains, 6 fusion architectures, and 2 missing-data modes over 125,000 samples. Through this benchmark, we find that architecture family is the strongest predictor of robustness, outweighing parameter count. Channel-independent models tolerate modality missing well but can be sensitive to within-modality missing, especially on short sequences. Curriculum modality dropout protects reliably only up to the maximum dropout rate used in training. We also find that channel count, sequence length, and modality alignment jointly determine which failure mode poses the greater threat. Finally, a PTB-XL case study suggests that diffusion-based imputation can improve downstream classification under within-modality missing, with the largest gains for models whose expert routing is most sensitive to corrupted inputs, though broader validation across datasets remains an open direction. MuteBench provides practitioners with concrete guidance for both selecting existing architectures and informing the design of future robust multimodal fusion methods.

Related papers

• Conditional Imputation for Within-Modality Missingness in Multi-Modal Federated Learning [8.35064965629059]
  Multimodal Federated Learning (MMFL) enables privacy-preserving collaborative training.<n>Real-world clinical applications often suffer from within-modality missingness caused by sensor intermittency or irregular sampling.<n>We propose CondI, a framework explicitly addressing this missingness using conditional diffusion models.
  arXiv  Detail & Related papers  (2026-04-25T02:35:08Z)
• Architecture-Agnostic Modality-Isolated Gated Fusion for Robust Multi-Modal Prostate MRI Segmentation [0.0]
  Multi-weighted prostate MRI combines T2-parametric (T2W), apparent diffusion coefficient (ADC), and high b-value diffusion-weighted (HBV) sequences.<n>In practice, the diffusion sequences are more frequently subject to acquisition variability, motion, and artifacts than T2W.<n>We propose Modality-Isolated Gated Fusion (MIGF) to maintain separate modality-specific encoding streams before a learned gating stage.
  arXiv  Detail & Related papers  (2026-04-12T15:54:21Z)
• Missing-Aware Multimodal Fusion for Unified Microservice Incident Management [22.703696384749595]
  ARMOR is a self-supervised framework designed for missing modality scenarios.<n>It employs self-supervised auto-regression with mask-guided reconstruction to jointly optimize anomaly detection, failure triage, and root cause localization.<n>It achieves state-of-the-art performance under complete data conditions and maintains robust diagnostic accuracy even with severe modality loss.
  arXiv  Detail & Related papers  (2026-03-26T15:14:57Z)
• BMDS-Net: A Bayesian Multi-Modal Deep Supervision Network for Robust Brain Tumor Segmentation [21.538098924595754]
  We propose BMDS-Net, a unified framework that prioritizes clinical robustness and trustworthiness over simple metric.<n>Our contribution is three-fold. First, we construct a robust deterministic backbone by integrating a Zero-Init Multimodal Conmodal Fusion (MMCF) module and a Residual-Gated Deep Decoder Supervision (DDS) mechanism.<n>Second, we introduce a memory-efficient Bayesian fine-tuning strategy that transforms the network into a probabilistic predictor, providing voxel-wise uncertainty maps.<n>Third, comprehensive experiments on the BraTS 2021 dataset demonstrate that BMDS-Net not only maintains competitive accuracy
  arXiv  Detail & Related papers  (2026-01-24T16:06:43Z)
• A Flow Model with Low-Rank Transformers for Incomplete Multimodal Survival Analysis [36.102030480314816]
  We propose a novel framework that combines a low-rank Transformer with a flow-based generative model for robust and flexible multimodal survival prediction.<n>Our method not only achieves state-of-the-art performance under complete modality settings, but also maintains robust and superior accuracy under the incomplete modalities scenario.
  arXiv  Detail & Related papers  (2025-10-22T02:27:05Z)
• ShortcutBreaker: Low-Rank Noisy Bottleneck with Global Perturbation Attention for Multi-Class Unsupervised Anomaly Detection [59.89803740308262]
  ShortcutBreaker is a novel unified feature-reconstruction framework for MUAD tasks.<n>It features two key innovations to address the issue of shortcuts.<n>The proposed method achieves a remarkable image-level AUROC of 99.8%, 98.9%, 90.6%, and 87.8% on four datasets.
  arXiv  Detail & Related papers  (2025-10-21T06:51:30Z)
• Hierarchical Self-Supervised Representation Learning for Depression Detection from Speech [51.14752758616364]
  Speech-based depression detection (SDD) is a promising, non-invasive alternative to traditional clinical assessments.<n>We propose HAREN-CTC, a novel architecture that integrates multi-layer SSL features using cross-attention within a multitask learning framework.<n>The model achieves state-of-the-art macro F1-scores of 0.81 on DAIC-WOZ and 0.82 on MODMA, outperforming prior methods across both evaluation scenarios.
  arXiv  Detail & Related papers  (2025-10-05T09:32:12Z)
• Cross-Attention is Not Enough: Incongruity-Aware Dynamic Hierarchical Fusion for Multimodal Affect Recognition [69.32305810128994]
  Incongruity between modalities poses a challenge for multimodal fusion, especially in affect recognition. We propose the Hierarchical Crossmodal Transformer with Dynamic Modality Gating (HCT-DMG), a lightweight incongruity-aware model. HCT-DMG: 1) outperforms previous multimodal models with a reduced size of approximately 0.8M parameters; 2) recognizes hard samples where incongruity makes affect recognition difficult; 3) mitigates the incongruity at the latent level in crossmodal attention.
  arXiv  Detail & Related papers  (2023-05-23T01:24:15Z)
• Enhancing Multiple Reliability Measures via Nuisance-extended Information Bottleneck [77.37409441129995]
  In practical scenarios where training data is limited, many predictive signals in the data can be rather from some biases in data acquisition. We consider an adversarial threat model under a mutual information constraint to cover a wider class of perturbations in training. We propose an autoencoder-based training to implement the objective, as well as practical encoder designs to facilitate the proposed hybrid discriminative-generative training.
  arXiv  Detail & Related papers  (2023-03-24T16:03:21Z)
• Cascaded Multi-Modal Mixing Transformers for Alzheimer's Disease Classification with Incomplete Data [8.536869574065195]
  Multi-Modal Mixing Transformer (3MAT) is a disease classification transformer that not only leverages multi-modal data but also handles missing data scenarios. We propose a novel modality dropout mechanism to ensure an unprecedented level of modality independence and robustness to handle missing data scenarios.
  arXiv  Detail & Related papers  (2022-10-01T11:31:02Z)
• Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysis [96.46952672172021]
  Bi-Bimodal Fusion Network (BBFN) is a novel end-to-end network that performs fusion on pairwise modality representations. Model takes two bimodal pairs as input due to known information imbalance among modalities.
  arXiv  Detail & Related papers  (2021-07-28T23:33:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.