BALM: A Model-Agnostic Framework for Balanced Multimodal Learning under Imbalanced Missing Rates

• URL: http://arxiv.org/abs/2603.19718v1
• Date: Fri, 20 Mar 2026 07:47:09 GMT
• Title: BALM: A Model-Agnostic Framework for Balanced Multimodal Learning under Imbalanced Missing Rates
• Abstract summary: BALM is a model-agnostic plug-in framework to achieve balanced multimodal learning under IMR.<n>We show that BALM consistently enhances robustness and improves performance under diverse missing and imbalance settings.
• Score: 1.9650181409005112
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Learning from multiple modalities often suffers from imbalance, where information-rich modalities dominate optimization while weaker or partially missing modalities contribute less. This imbalance becomes severe in realistic settings with imbalanced missing rates (IMR), where each modality is absent with different probabilities, distorting representation learning and gradient dynamics. We revisit this issue from a training-process perspective and propose BALM, a model-agnostic plug-in framework to achieve balanced multimodal learning under IMR. The framework comprises two complementary modules: the Feature Calibration Module (FCM), which recalibrates unimodal features using global context to establish a shared representation basis across heterogeneous missing patterns; the Gradient Rebalancing Module (GRM), which balances learning dynamics across modalities by modulating gradient magnitudes and directions from both distributional and spatial perspectives. BALM can be seamlessly integrated into diverse backbones, including multimodal emotion recognition (MER) models, without altering their architectures. Experimental results across multiple MER benchmarks confirm that BALM consistently enhances robustness and improves performance under diverse missing and imbalance settings. Code available at: https://github.com/np4s/BALM_CVPR2026.git

Related papers

• Model-Agnostic Meta Learning for Class Imbalance Adaptation [3.2488051786523813]
  Class imbalance is a widespread challenge in NLP tasks, hindering robust performance across diverse domains and applications.<n>We introduce Hardness-Aware Meta-Resample (HAMR), a unified framework that adaptively addresses both class imbalance and data difficulty.
  arXiv  Detail & Related papers  (2026-04-20T19:07:56Z)
• Mixture of Ranks with Degradation-Aware Routing for One-Step Real-World Image Super-Resolution [76.66229730098759]
  In real-world image super-resolution (Real-ISR), existing approaches mainly rely on fine-tuning pre-trained diffusion models.<n>We propose a Mixture-of-Ranks (MoR) architecture for single-step image super-resolution.<n>We introduce a fine-grained expert partitioning strategy that treats each rank in LoRA as an independent expert.
  arXiv  Detail & Related papers  (2025-11-20T04:11:44Z)
• Representation Space Constrained Learning with Modality Decoupling for Multimodal Object Detection [3.556651853847748]
  This paper presents a theoretical investigation of fusion degradation in multimodal detection.<n>It proposes a Representation Space Constrained Learning with Modality Decoupling (RSC-MD) method, which consists of two modules.<n>The proposed method effectively alleviates fusion degradation and achieves state-of-the-art performance across multiple benchmarks.
  arXiv  Detail & Related papers  (2025-11-19T13:41:27Z)
• An Integrated Fusion Framework for Ensemble Learning Leveraging Gradient Boosting and Fuzzy Rule-Based Models [59.13182819190547]
  Fuzzy rule-based models excel in interpretability and have seen widespread application across diverse fields.<n>They face challenges such as complex design specifications and scalability issues with large datasets.<n>This paper proposes an Integrated Fusion Framework that merges the strengths of both paradigms to enhance model performance and interpretability.
  arXiv  Detail & Related papers  (2025-11-11T10:28:23Z)
• Quantifying Multimodal Imbalance: A GMM-Guided Adaptive Loss for Audio-Visual Learning [12.236332735708473]
  Existing solutions mainly focus on optimization- or data-based strategies but rarely exploit the information inherent in multimodal imbalance.<n>We propose a novel quantitative analysis framework for Multimodal Imbalance and design a sample-level adaptive loss function.
  arXiv  Detail & Related papers  (2025-10-20T15:42:43Z)
• BTW: A Non-Parametric Variance Stabilization Framework for Multimodal Model Integration [20.600001069987318]
  We propose Beyond Two-modality Weighting (BTW) to dynamically adjust modality importance during training.<n>BTW computes per-example KL weights by measuring the divergence between each unimodal and the current multimodal prediction.<n>Our method significantly improves regression performance and multiclass classification accuracy.
  arXiv  Detail & Related papers  (2025-08-25T23:00:38Z)
• Modality Equilibrium Matters: Minor-Modality-Aware Adaptive Alternating for Cross-Modal Memory Enhancement [13.424541949553964]
  We propose a Shapley-guided alternating training framework that adaptively prioritizes minor modalities to balance and thus enhance the fusion.<n>We evaluate the performance in both balance and accuracy across four multimodal benchmark datasets, where our method achieves state-of-the-art (SOTA) results.
  arXiv  Detail & Related papers  (2025-05-26T02:02:57Z)
• Classifier-guided Gradient Modulation for Enhanced Multimodal Learning [50.7008456698935]
  Gradient-Guided Modulation (CGGM) is a novel method to balance multimodal learning with gradients. We conduct extensive experiments on four multimodal datasets: UPMC-Food 101, CMU-MOSI, IEMOCAP and BraTS. CGGM outperforms all the baselines and other state-of-the-art methods consistently.
  arXiv  Detail & Related papers  (2024-11-03T02:38:43Z)
• On-the-fly Modulation for Balanced Multimodal Learning [53.616094855778954]
  Multimodal learning is expected to boost model performance by integrating information from different modalities. The widely-used joint training strategy leads to imbalanced and under-optimized uni-modal representations. We propose On-the-fly Prediction Modulation (OPM) and On-the-fly Gradient Modulation (OGM) strategies to modulate the optimization of each modality.
  arXiv  Detail & Related papers  (2024-10-15T13:15:50Z)
• Improving Discriminative Multi-Modal Learning with Large-Scale Pre-Trained Models [51.5543321122664]
  This paper investigates how to better leverage large-scale pre-trained uni-modal models to enhance discriminative multi-modal learning. We introduce Multi-Modal Low-Rank Adaptation learning (MMLoRA)
  arXiv  Detail & Related papers  (2023-10-08T15:01:54Z)
• Meta-Causal Feature Learning for Out-of-Distribution Generalization [71.38239243414091]
  This paper presents a balanced meta-causal learner (BMCL), which includes a balanced task generation module (BTG) and a meta-causal feature learning module (MCFL) BMCL effectively identifies the class-invariant visual regions for classification and may serve as a general framework to improve the performance of the state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-08-22T09:07:02Z)
• Balanced Multimodal Learning via On-the-fly Gradient Modulation [10.5602074277814]
  Multimodal learning helps to comprehensively understand the world, by integrating different senses. We propose on-the-fly gradient modulation to adaptively control the optimization of each modality, via monitoring the discrepancy of their contribution towards the learning objective.
  arXiv  Detail & Related papers  (2022-03-29T08:26:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.