Enhancing Multi-Modal LLMs Reasoning via Difficulty-Aware Group Normalization

• URL: http://arxiv.org/abs/2602.21743v2
• Date: Thu, 26 Feb 2026 06:20:05 GMT
• Title: Enhancing Multi-Modal LLMs Reasoning via Difficulty-Aware Group Normalization
• Abstract summary: We propose difficulty-aware group normalization (Durian), which re-groups samples by difficulty levels and shares within each group.<n>Our approach preserves intra-group distinctions while eliminating sensitivity to extreme cases, yielding significant performance gains across multiple multimodal reasoning benchmarks.
• Score: 38.26061472669552
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) and Group Relative Policy Optimization (GRPO) have significantly advanced the reasoning capabilities of large language models. Extending these methods to multimodal settings, however, faces a critical challenge: the instability of std-based normalization, which is easily distorted by extreme samples with nearly positive or negative rewards. Unlike pure-text LLMs, multimodal models are particularly sensitive to such distortions, as both perceptual and reasoning errors influence their responses. To address this, we characterize each sample by its difficulty, defined through perceptual complexity (measured via visual entropy) and reasoning uncertainty (captured by model confidence). Building on this characterization, we propose difficulty-aware group normalization (Durian), which re-groups samples by difficulty levels and shares the std within each group. Our approach preserves GRPO's intra-group distinctions while eliminating sensitivity to extreme cases, yielding significant performance gains across multiple multimodal reasoning benchmarks.

Related papers

• Clustered Self-Assessment: A Simple yet Effective Method for Uncertainty Quantification in Large Language Models [44.59799077650502]
  We propose a simple yet effective self-assessment method for uncertainty quantification in large language models.<n>Our approach groups sampled generations into semantically distinct clusters, converts them into answer options in a structured multiple-choice question, and uses the probability assigned by the model to each option as a confidence estimate.
  arXiv  Detail & Related papers  (2026-06-02T16:25:54Z)
• Too Correct to Learn: Reinforcement Learning on Saturated Reasoning Data [55.84428098924793]
  We propose Constrained Uniform Top-K Sampling (CUTS), a parameter-free decoding strategy enforcing structure-preserving exploration.<n>We integrate this into Mixed-CUTS, a training framework synergizing exploitative and exploratory rollouts to amplify intra-group advantage variance.<n> Notably, Mixed-CUTS improves Pass@1 accuracy on the challenging AIME25 benchmark by up to 15.1% over standard GRPO.
  arXiv  Detail & Related papers  (2026-04-20T16:43:28Z)
• BROTHER: Behavioral Recognition Optimized Through Heterogeneous Ensemble Regularization for Ambivalence and Hesitancy [39.407739937584104]
  Ambivalence and Hesitancy (A/H) in naturalistic video settings remains a significant challenge in affective computing.<n>We propose a highly regularized, multimodal fusion pipeline to predict A/H at the video level.
  arXiv  Detail & Related papers  (2026-03-15T12:53:41Z)
• DIVA-GRPO: Enhancing Multimodal Reasoning through Difficulty-Adaptive Variant Advantage [83.64031699341862]
  Reinforcement learning with group relative policy optimization ( GRPO) has become a widely adopted approach for enhancing the reasoning capabilities of multimodal large language models (MLLMs)<n> GRPO enables long-chain reasoning without a critic, but it often suffers from sparse rewards on difficult problems and advantage vanishing when group-level rewards are too consistent for overly easy or hard problems.<n>We propose DIVA-GRPO, a difficulty-adaptive variant advantage method that adjusts variant difficulty distributions from a global perspective.
  arXiv  Detail & Related papers  (2026-03-01T13:47:35Z)
• Harnessing Consistency for Robust Test-Time LLM Ensemble [88.55393815158608]
  CoRE is a plug-and-play technique that harnesses model consistency for robust LLM ensemble.<n> Token-level consistency captures fine-grained disagreements by applying a low-pass filter to downweight uncertain tokens.<n>Model-level consistency models global agreement by promoting model outputs with high self-confidence.
  arXiv  Detail & Related papers  (2025-10-12T04:18:45Z)
• Beyond Exponential Decay: Rethinking Error Accumulation in Large Language Models [0.0]
  We show that errors are not uniformly distributed but are concentrated at sparse "key tokens" representing critical decision junctions.<n>We propose a framework for next-generation systems centered on selective preservation of semantically vital tokens.
  arXiv  Detail & Related papers  (2025-05-30T03:57:31Z)
• Visual Perturbation and Adaptive Hard Negative Contrastive Learning for Compositional Reasoning in Vision-Language Models [16.405694961196925]
  Vision-Language Models (VLMs) are essential for multimodal tasks, especially compositional reasoning (CR) tasks.<n>Existing methods primarily fine-tune the model by generating text-based hard negative samples.<n>AHNPL translates text-based hard negatives into the visual domain to generate semantically disturbed image-based negatives for training the model.
  arXiv  Detail & Related papers  (2025-05-21T14:28:43Z)
• Language Model Cascades: Token-level uncertainty and beyond [65.38515344964647]
  Recent advances in language models (LMs) have led to significant improvements in quality on complex NLP tasks. Cascading offers a simple strategy to achieve more favorable cost-quality tradeoffs. We show that incorporating token-level uncertainty through learned post-hoc deferral rules can significantly outperform simple aggregation strategies.
  arXiv  Detail & Related papers  (2024-04-15T21:02:48Z)
• Modeling the Q-Diversity in a Min-max Play Game for Robust Optimization [61.39201891894024]
  Group distributionally robust optimization (group DRO) can minimize the worst-case loss over pre-defined groups. We reformulate the group DRO framework by proposing Q-Diversity. Characterized by an interactive training mode, Q-Diversity relaxes the group identification from annotation into direct parameterization.
  arXiv  Detail & Related papers  (2023-05-20T07:02:27Z)
• Enhancing Chain-of-Thoughts Prompting with Iterative Bootstrapping in Large Language Models [81.01397924280612]
  Large language models (LLMs) can achieve highly effective performance on various reasoning tasks by incorporating step-by-step chain-of-thought (CoT) prompting as demonstrations. We introduce Iter-CoT (Iterative bootstrapping in Chain-of-Thoughts Prompting), an iterative bootstrapping approach for selecting exemplars and generating reasoning chains.
  arXiv  Detail & Related papers  (2023-04-23T13:54:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.