QA-MoE: Towards a Continuous Reliability Spectrum with Quality-Aware Mixture of Experts for Robust Multimodal Sentiment Analysis

• URL: http://arxiv.org/abs/2604.05704v2
• Date: Wed, 08 Apr 2026 05:42:25 GMT
• Title: QA-MoE: Towards a Continuous Reliability Spectrum with Quality-Aware Mixture of Experts for Robust Multimodal Sentiment Analysis
• Abstract summary: We introduce a Continuous Reliability Spectrum to unify missingness and quality degradation into a single framework.<n>We then propose QA-MoE, a Quality-Aware Mixture-of-Experts framework that quantifies modality reliability via self-supervised aleatoric uncertainty.<n>Extensive experiments indicate that QA-MoE achieves competitive or state-of-the-art performance across diverse degradation scenarios.
• Score: 9.551550608808752
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal Sentiment Analysis (MSA) aims to infer human sentiment from textual, acoustic, and visual signals. In real-world scenarios, however, multimodal inputs are often compromised by dynamic noise or modality missingness. Existing methods typically treat these imperfections as discrete cases or assume fixed corruption ratios, which limits their adaptability to continuously varying reliability conditions. To address this, we first introduce a Continuous Reliability Spectrum to unify missingness and quality degradation into a single framework. Building on this, we propose QA-MoE, a Quality-Aware Mixture-of-Experts framework that quantifies modality reliability via self-supervised aleatoric uncertainty. This mechanism explicitly guides expert routing, enabling the model to suppress error propagation from unreliable signals while preserving task-relevant information. Extensive experiments indicate that QA-MoE achieves competitive or state-of-the-art performance across diverse degradation scenarios and exhibits a promising One-Checkpoint-for-All property in practice.

Related papers

• Rethinking Modality Reliability in Multimodal Sentiment Analysis with Incomplete Observations [9.993152436695421]
  We argue that modality reliability is a central variable in incomplete-observation settings. Failure to model it explicitly gives rise to two related issues.<n>We propose Modality Reliability-Calibrated Framework for Multimodal Sentiment Analysis with incomplete observations.<n> MRCF contains a Reliability-Aware Branch that estimates sample-specific modality reliability from intramodal quality cues and cross-modal semantic consistency.<n>Experiments on CMU-MOSI, CMU-MOSEI, and CH-SIMS show that MRCF achieves strong performance under standard incomplete-observation protocols.
  arXiv  Detail & Related papers  (2026-08-04T13:02:48Z)
• MRUF: Multi-granularity Routing with Uncertainty-Aware Fusion for Robust Multimodal Sentiment Analysis [20.73891715749963]
  We propose a reliability-aware fusion method that combines multi-granularity routing with uncertainty-aware calibration.<n> MRUF summarizes sentiment-relevant representations, performs subspace- and modality-level routing, and supervises modality routing with leave-one-out error increases to estimate utterance-level modality importance.
  arXiv  Detail & Related papers  (2026-07-12T06:32:54Z)
• AnchorMoE: Interpretable Time Series Classification via Anchor-Routed MoE [51.994752158953084]
  AnchorMoE is an interpretable-by-construction classification framework.<n>It encodes multi-view representations of local patches and routes them to specialized experts.<n>Experiments on real-world and synthetic benchmarks demonstrate that AnchorMoE achieves highly competitive classification performance.
  arXiv  Detail & Related papers  (2026-06-02T13:30:54Z)
• Towards Trustworthy Depression Estimation via Disentangled Evidential Learning [50.22167852149165]
  We propose EviDep, an evidential learning framework that jointly quantifies depression severity.<n>EviDep enforces strict information integrity to guarantee robust evidence synthesis.<n>It achieves state-of-the-art predictive accuracy and superior uncertainty calibration, delivering a robust fail-safe mechanism for trustworthy clinical screening.
  arXiv  Detail & Related papers  (2026-04-17T13:27:11Z)
• Stage-Adaptive Reliability Modeling for Continuous Valence-Arousal Estimation [11.761508894410182]
  We propose a Stage-Adaptive reliability modeling framework that explicitly estimates and calibrates modality-wise confidence during multimodal integration.<n>SAGE introduces a reliability-aware fusion mechanism that dynamically rebalances audio and visual representations according to their stage-dependent informativeness.<n>Experiments on the Aff-Wild2 benchmark demonstrate that SAGE consistently improves concordance correlation coefficient scores compared with existing multimodal fusion approaches.
  arXiv  Detail & Related papers  (2026-03-12T02:45:41Z)
• Progressive Representation Learning for Multimodal Sentiment Analysis with Incomplete Modalities [26.613566093134065]
  Multimodal Sentiment Analysis seeks to infer human emotions by integrating textual, acoustic, and visual cues.<n>Existing approaches often rely on all modalities are completeness, whereas real-world applications frequently encounter noise, hardware failures, or privacy restrictions that result in missing modalities.<n>We propose PRLF, a Progressive Representation Learning Framework designed for MSA under uncertain missing-modality conditions.
  arXiv  Detail & Related papers  (2026-03-10T02:45:02Z)
• Adaptive Confidence Regularization for Multimodal Failure Detection [21.36399203050946]
  We propose Adaptive Confidence Regularization (ACR), a framework specifically designed to detect multimodal failures.<n>Our approach is driven by a key observation: in most failure cases, the confidence of the multimodal prediction is significantly lower than that of at least one unimodal branch.<n>In addition, we propose Multimodal Feature Swapping, a novel outlier technique that generates challenging, failure-aware training examples.
  arXiv  Detail & Related papers  (2026-03-02T18:56:38Z)
• BriMA: Bridged Modality Adaptation for Multi-Modal Continual Action Quality Assessment [25.689906499244533]
  Action Quality Assessment (AQA) aims to score how well an action is performed and is widely used in sports analysis, rehabilitation assessment, and human skill evaluation.<n>We introduce Bridged Modality Adaptation (BriMA), an innovative approach to multi-modal continual AQA under modality-missing conditions.<n>BriMA consists of a memory-guided bridging imputation module that reconstructs missing modalities using both task-agnostic and task-specific representations, and a modality-aware replay mechanism that prioritizes informative samples based on modality distortion and distribution drift.
  arXiv  Detail & Related papers  (2026-02-22T13:00:52Z)
• Agentic Confidence Calibration [67.50096917021521]
  Holistic Trajectory (HTC) is a novel diagnostic framework for AI agents.<n>HTC consistently surpasses strong baselines in both calibration and discrimination.<n>HTC provides interpretability by revealing the signals behind failure.
  arXiv  Detail & Related papers  (2026-01-22T09:08:25Z)
• Test-time Adaptive Hierarchical Co-enhanced Denoising Network for Reliable Multimodal Classification [55.56234913868664]
  We propose Test-time Adaptive Hierarchical Co-enhanced Denoising Network (TAHCD) for reliable learning on multimodal data.<n>The proposed method achieves superior classification performance, robustness, and generalization compared with state-of-the-art reliable multimodal learning approaches.
  arXiv  Detail & Related papers  (2026-01-12T03:14:12Z)
• Uncertainty-Resilient Multimodal Learning via Consistency-Guided Cross-Modal Transfer [0.0]
  This thesis explores uncertainty-resilient multimodal learning through consistency-guided cross-modal transfer.<n>The central idea is to use cross-modal semantic consistency as a basis for robust representation learning.<n>Building on this foundation, the thesis investigates strategies to enhance semantic robustness, improve data efficiency, and reduce the impact of noise and imperfect supervision.
  arXiv  Detail & Related papers  (2025-11-18T15:26:42Z)
• Multi-QuAD: Multi-Level Quality-Adaptive Dynamic Network for Reliable Multimodal Classification [57.08108545219043]
  Existing reliable multimodal classification methods fail to provide robust estimation of data quality.<n>New framework for reliable classification termed textitMulti-level Quality-Adaptive Dynamic multimodal network (Multi-QuAD) is proposed.<n>Experiments conducted on four datasets demonstrate that Multi-QuAD significantly outperforms state-of-the-art methods in classification performance and reliability.
  arXiv  Detail & Related papers  (2024-12-19T03:26:51Z)
• Uncertainty-Driven Action Quality Assessment [11.958132175629368]
  We propose a novel probabilistic model, named Uncertainty-Driven AQA (UD-AQA), to capture the diversity among multiple judge scores.<n>We generate the estimation of uncertainty for each prediction, which is employed to re-weight AQA regression loss.<n>Our proposed method achieves competitive results on three benchmarks including the Olympic events MTL-AQA and FineDiving, and the surgical skill JIGSAWS datasets.
  arXiv  Detail & Related papers  (2022-07-29T07:21:15Z)
• Trusted Multi-View Classification with Dynamic Evidential Fusion [73.35990456162745]
  We propose a novel multi-view classification algorithm, termed trusted multi-view classification (TMC) TMC provides a new paradigm for multi-view learning by dynamically integrating different views at an evidence level. Both theoretical and experimental results validate the effectiveness of the proposed model in accuracy, robustness and trustworthiness.
  arXiv  Detail & Related papers  (2022-04-25T03:48:49Z)
• Trustworthy Multimodal Regression with Mixture of Normal-inverse Gamma Distributions [91.63716984911278]
  We introduce a novel Mixture of Normal-Inverse Gamma distributions (MoNIG) algorithm, which efficiently estimates uncertainty in principle for adaptive integration of different modalities and produces a trustworthy regression result. Experimental results on both synthetic and different real-world data demonstrate the effectiveness and trustworthiness of our method on various multimodal regression tasks.
  arXiv  Detail & Related papers  (2021-11-11T14:28:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.