Teaching MLLMs to Say No: Generalized Referring Expression Comprehension via Refusal Calibrated GRPO

• URL: http://arxiv.org/abs/2608.04698v1
• Date: Wed, 05 Aug 2026 11:07:22 GMT
• Title: Teaching MLLMs to Say No: Generalized Referring Expression Comprehension via Refusal Calibrated GRPO
• Abstract summary: Generalized Referring Expression (GREC) requires a model to localize the object described by a textual expression when it exists (positive sample) and to refuse output when it does not (negative sample)<n>Existing post-training approaches such as supervised fine-tuning (SFT) and reinforcement learning (RL) enhance refusal behavior but usually degrade localization accuracy on positive samples, undermining the model's core competence.<n>We propose Refusal-Calibrated Group Relative Policy Optimization (RC-GRPO), a calibrated RL strategy that strengthens the refusal ability of MLLMs while preserving localization performance.
• Score: 8.477600107706547
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We tackle the challenging yet underexplored task of Generalized Referring Expression Comprehension (GREC), which requires a model to localize the object described by a textual expression when it exists (positive sample) and to refuse output when it does not (negative sample). Although Multimodal Large Language Models (MLLMs) excel at localizing existing objects, they often fail to reject nonexistent ones due to the absence of negative samples during training, producing hallucinated bounding boxes. Existing post-training approaches such as supervised fine-tuning (SFT) and reinforcement learning (RL) enhance refusal behavior but usually degrade localization accuracy on positive samples, undermining the model's core competence. To address this, we propose Refusal-Calibrated Group Relative Policy Optimization (RC-GRPO), a calibrated RL strategy that strengthens the refusal ability of MLLMs while preserving localization performance. It enforces "None" outputs in rollouts for valid advantage estimation on negative samples and applies a penalty to prevent over-refusal on positives, achieving a balanced trade-off between accuracy and reliability. A second-stage reasoning reinforcement further consolidates causal understanding and interpretability. Experiments on three GREC benchmarks demonstrate that RC-GRPO attains superior localization accuracy while maintaining strong refusal capability.

Related papers

• Inference-Time Steering for Cross-Lingual Factual Consistency in LLMs [51.56484100374058]
  Large Language Models (LLMs) demonstrate remarkable multilingual fluency, but their internal knowledge representations remain disproportionately biased toward high-resource languages.<n>We investigate whether these biases can be mitigated at inference time, forcing an English-prompted model to answer as if it were queried in target languages.<n>Experiments on Gemma 3 12B Instruct reveal persona prompting to be the strongest overall intervention, balancing efficacy, safety, and out-of-domain generalization.
  arXiv  Detail & Related papers  (2026-07-21T16:15:05Z)
• Estimating Rare Events in Language Models with Proper Evaluation [16.1901795614153]
  We introduce Gradient Activation Adaptive Multi-Level Splitting (GA-AMLS)<n>GA-AMLS adapts rare-event Monte Carlo methods to the continuous activation space of language models.<n>We also propose the Shifted-Power Bregman (SPB) Loss, a proper scoring rule that remains finite for zero-estimates.
  arXiv  Detail & Related papers  (2026-07-20T19:07:43Z)
• Enhancing Rubric-based RL via Self-Distillation [23.111278687216544]
  We propose Criterion-Distilled Policy Optimization (CriPO), which enhances rubric-based RL via on-policy self-distillation.<n>CriPO consistently outperforms rubric-based RL, achieving stronger final performance with approximately $2times$ fewer optimization steps.
  arXiv  Detail & Related papers  (2026-07-20T15:50:02Z)
• Interactive Critique-Revision Training for Reliable Structured LLM Generation [18.00222080273147]
  We propose DPA-GRPO, a paired-action training method for a two-player generator--verifier game with structured verifier interventions.<n>We analyze the unregularized game and show that positive probability on strictly lower-reward intervention or revision actions creates a profitable unilateral deviation.<n>Experiments on TaxCalc TY24 show that DPA-GRPO improves structured decision accuracy over zero-shot generation and generator-only RL baselines.
  arXiv  Detail & Related papers  (2026-05-08T17:00:38Z)
• Too Correct to Learn: Reinforcement Learning on Saturated Reasoning Data [55.84428098924793]
  We propose Constrained Uniform Top-K Sampling (CUTS), a parameter-free decoding strategy enforcing structure-preserving exploration.<n>We integrate this into Mixed-CUTS, a training framework synergizing exploitative and exploratory rollouts to amplify intra-group advantage variance.<n> Notably, Mixed-CUTS improves Pass@1 accuracy on the challenging AIME25 benchmark by up to 15.1% over standard GRPO.
  arXiv  Detail & Related papers  (2026-04-20T16:43:28Z)
• SERC: LDPC-Inspired Semantic Error Correction for Retrieval-Augmented Generation [3.53018552538156]
  Large Language Models (LLMs) have demonstrated remarkable capabilities, but their reliability is significantly compromised by hallucinations.<n>We propose the LDPC-inspired semantic error correction for retrieval-augmented generation (SERC)<n>We reformulate the text generation process as a semantic noisy channel, treating generated responses as noise-corrupted codewords.
  arXiv  Detail & Related papers  (2026-04-12T09:09:46Z)
• WS-GRPO: Weakly-Supervised Group-Relative Policy Optimization for Rollout-Efficient Reasoning [67.45237332694025]
  Group Relative Policy Optimization is effective for training language models on complex reasoning.<n>We propose Weakly Supervised GRPO, which improves rollout efficiency by converting terminal rewards into correctness-aware guidance.
  arXiv  Detail & Related papers  (2026-02-19T02:43:35Z)
• Uncalibrated Reasoning: GRPO Induces Overconfidence for Stochastic Outcomes [55.2480439325792]
  Reinforcement learning (RL) has proven remarkably effective at improving the accuracy of language models in verifiable and deterministic domains like mathematics.<n>Here, we examine if current RL methods are also effective at optimizing language models in verifiable domains with outcomes, like scientific experiments.
  arXiv  Detail & Related papers  (2025-08-15T20:50:53Z)
• Flipping Against All Odds: Reducing LLM Coin Flip Bias via Verbalized Rejection Sampling [59.133428586090226]
  Large language models (LLMs) can often accurately describe probability distributions using natural language.<n>This mismatch limits their use in tasks requiring reliableity, such as Monte Carlo methods, agent-based simulations, and randomized decision-making.<n>We introduce Verbalized Rejection Sampling (VRS), a natural-language adaptation of classical rejection sampling.
  arXiv  Detail & Related papers  (2025-06-11T17:59:58Z)
• A Minimalist Approach to LLM Reasoning: from Rejection Sampling to Reinforce [68.99924691391048]
  We revisit GRPO from a reinforce-like algorithm perspective and analyze its core components.<n>We find that a simple rejection sampling baseline, RAFT, yields competitive performance than GRPO and PPO.<n>Motivated by this insight, we propose Reinforce-Rej, a minimal extension of policy gradient that filters both entirely incorrect and entirely correct samples.
  arXiv  Detail & Related papers  (2025-04-15T16:15:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.