BalCapRL: A Balanced Framework for RL-Based MLLM Image Captioning

• URL: http://arxiv.org/abs/2605.07394v1
• Date: Fri, 08 May 2026 07:48:36 GMT
• Title: BalCapRL: A Balanced Framework for RL-Based MLLM Image Captioning
• Abstract summary: We propose a more balanced reinforcement learning framework that jointly optimize utility-aware correctness, reference coverage, and linguistic quality.<n>We show that our method consistently improves caption quality, with peak gains of +13.6 DCScore, +9.0 CaptionQA, and +29.0 Cap across different models.
• Score: 4.441850776689605
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image captioning is one of the most fundamental tasks in computer vision. Owing to its open-ended nature, it has received significant attention in the era of multimodal large language models (MLLMs). In pursuit of ever more detailed and accurate captions, recent work has increasingly turned to reinforcement learning (RL). However, existing captioning-RL methods and evaluation metrics often emphasize a narrow notion of caption quality, inducing trade-offs across core dimensions of captioning. For example, utility-oriented objectives can encourage noisy, hallucinated, or overlong captions that improve downstream question answering while harming fluency, whereas arena-style objectives can favor fluent but generic descriptions with limited usefulness. To address this, we propose a more balanced RL framework that jointly optimizes utility-aware correctness, reference coverage, and linguistic quality. In order to effectively optimize the resulting continuous multi-objective reward formulation, we apply GDPO-style reward-decoupled normalization to continuous-valued captioning rewards and show that it improves performance over vanilla GRPO. Additionally, we introduce length-conditional reward masking, yielding a more suitable length penalty for captioning. Across LLaVA-1.5-7B and Qwen2.5-VL 3B and 7B base models, our method consistently improves caption quality, with peak gains of +13.6 DCScore, +9.0 CaptionQA, and +29.0 CapArena across different models.

Related papers

• Re$^3$Cap: Retrieval-Guided Refinement for Image Captioning Enhancement via Reinforcement Learning [13.794868838200465]
  We argue that multi-modal retrieval can serve as an effective reasoning signal for caption refinement.<n>We present the Retrieval-Guided Refinement for Image Captioning (Re$3$Cap), a retrieval-guided reasoning strategy that enhances image captioning without requiring additional annotations.
  arXiv  Detail & Related papers  (2026-08-21T17:07:41Z)
• CapRL++: Unified Reinforcement Learning with Verifiable Rewards for Dense Image and Video Captioning [86.9494763617273]
  We introduce Captioning Reinforcement Learning++ (CapRL++), a reference-free training framework that redefines caption quality through its utility.<n>We show that CapRL++ improves dense caption quality and strengthens caption-based pretraining across tasks such as spatial and temporal understanding.
  arXiv  Detail & Related papers  (2026-06-08T12:09:20Z)
• VCap: Hypergeometric Rewards for Weak-to-Strong Visual Captioning [57.588999592609646]
  We propose VCap, a Witness-Adjudicator reward that pairs the reference caption (a witness) with the visual signal (an adjudicator)<n>VCap delivers a reward signal with hypergeometric-distribution-level precision for caption quality verification.<n>In our experiments, an 8B model trained with VCap outperforms open- and closed-source SOTA models on multiple image and video captioning benchmarks.
  arXiv  Detail & Related papers  (2026-05-27T06:27:04Z)
• ITIScore: An Image-to-Text-to-Image Rating Framework for the Image Captioning Ability of MLLMs [84.09282931360089]
  ICBench is a large-scale image captioning benchmark that covers 12 content categories and consists of both short and long captions on 2K images.<n>We conduct extensive human subjective studies to obtain mean opinion scores (MOSs) across fine-grained evaluation dimensions.<n>We propose an automated evaluation metric, textbfITIScore, based on an image-to-text-to-image framework, which measures caption quality through reconstruction consistency.
  arXiv  Detail & Related papers  (2026-04-04T15:33:07Z)
• RubiCap: Rubric-Guided Reinforcement Learning for Dense Image Captioning [15.881702574415861]
  RubiCap is a novelReinforcement learning framework that derives fine-grained, sample-specific reward signals from rubrics.<n> RubiCap achieves the highest win rates on CapArena, outperforming supervised distillation, prior RL methods, human-expert annotations, and GPT-4V-augmented outputs.
  arXiv  Detail & Related papers  (2026-03-10T03:51:27Z)
• CCCaption: Dual-Reward Reinforcement Learning for Complete and Correct Image Captioning [23.289413412387223]
  We introduce CCCaption: a dual-reward reinforcement learning framework with a dedicated fine-tuning corpus.<n>For completeness, we use diverse LVLMs to disentangle the image into a set of visual queries, and reward captions that answer more of these queries.<n>For correctness, we penalize captions that contain hallucinations by validating the authenticity of sub-caption queries.
  arXiv  Detail & Related papers  (2026-02-25T07:34:26Z)
• CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning [90.19455861166745]
  We introduce Captioning Reinforcement Learning (CapRL), a training framework that redefines caption quality through its utility.<n>As the first study to apply RLVR to the subjective image captioning task, we demonstrate that CapRL significantly enhances multiple settings.<n>CapRL achieves performance comparable to Qwen2.5-VL-72B, while exceeding the baseline by an average margin of 8.4%.
  arXiv  Detail & Related papers  (2025-09-26T17:59:55Z)
• One More Glance with Sharp Eyes: Rethinking Lightweight Captioning as a Practical Visual Specialist [58.89538703878721]
  We build lightweight captioning models using a 125M- parameter language model, 56 times smaller than LLaMA-7B.<n>We evaluate their performance on single-sentence but on detailed captioning tasks.<n>We develop a novel captioning framework, Sharp-Eyed Refinement, which enhances caption quality by refining coarse descriptions into more precise captions.
  arXiv  Detail & Related papers  (2025-08-29T09:29:27Z)
• Toward Robust Hyper-Detailed Image Captioning: A Multiagent Approach and Dual Evaluation Metrics for Factuality and Coverage [50.84150600032693]
  Multimodal large language models (MLLMs) excel at generating highly detailed captions but often produce hallucinations.<n>We propose a multiagent approach that leverages LLM-MLLM collaboration to correct given captions.<n>Our proposed method significantly enhances the factual accuracy of captions, even improving those generated by GPT-4V.
  arXiv  Detail & Related papers  (2024-12-20T01:37:22Z)
• Mitigating Open-Vocabulary Caption Hallucinations [33.960405731583656]
  We propose a framework for addressing hallucinations in image captioning in the open-vocabulary setting. Our framework includes a new benchmark, OpenCHAIR, that leverages generative foundation models to evaluate open-vocabulary object hallucinations. To mitigate open-vocabulary hallucinations without using a closed object list, we propose MOCHa.
  arXiv  Detail & Related papers  (2023-12-06T17:28:03Z)
• Improving Image Captioning Descriptiveness by Ranking and LLM-based Fusion [8.526212812623202]
  State-of-The-Art (SoTA) image captioning models are often trained on the MicroSoft Common Objects in Context dataset.<n>We present a novel approach to generate richer and more informative image captions by combining the captions generated from different SoTA captioning models.
  arXiv  Detail & Related papers  (2023-06-20T15:13:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.