CapRL++: Unified Reinforcement Learning with Verifiable Rewards for Dense Image and Video Captioning

• URL: http://arxiv.org/abs/2606.09393v1
• Date: Mon, 08 Jun 2026 12:09:20 GMT
• Title: CapRL++: Unified Reinforcement Learning with Verifiable Rewards for Dense Image and Video Captioning
• Abstract summary: We introduce Captioning Reinforcement Learning++ (CapRL++), a reference-free training framework that redefines caption quality through its utility.<n>We show that CapRL++ improves dense caption quality and strengthens caption-based pretraining across tasks such as spatial and temporal understanding.
• Score: 86.9494763617273
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image and video captioning are fundamental tasks that bridge the visual and linguistic domains, playing a critical role in pre-training Large Vision-Language Models (LVLMs). Current state-of-the-art captioning models are typically trained with Supervised Fine-Tuning (SFT), a paradigm that relies on expensive, non-scalable annotations and often causes models to memorize specific ground-truth answers, limiting their generality and ability to generate diverse, creative descriptions. To overcome these limitations, we propose applying Reinforcement Learning with Verifiable Rewards (RLVR) to the open-ended task of multimodal captioning. We introduce Captioning Reinforcement Learning++ (CapRL++), a novel reference-free training framework that redefines caption quality through its utility: a high-quality caption should enable a non-visual language model to accurately answer questions about the corresponding visual content. CapRL++ employs a decoupled two-stage pipeline where an LVLM generates a caption, and the objective reward is derived from the accuracy of a separate, vision-free LLM answering Multiple-Choice Questions based solely on that caption. Evaluations on more than 20 image and video benchmarks show that CapRL++ improves dense caption quality and strengthens caption-based pretraining across tasks such as spatial and temporal understanding. Pretraining on scalable image and video caption datasets annotated by CapRL++ yields substantial downstream gains. Furthermore, within the Prism Framework for caption quality evaluation, compact models trained with CapRL++ achieve dense captioning performance comparable to substantially larger models such as Qwen2.5-VL-72B and Qwen3-VL-235B-A22B. These results validate that CapRL++ effectively trains models to produce generalizable, high-fidelity descriptions, establishing a robust foundation beyond the limitations of traditional SFT.

Related papers

• VCap: Hypergeometric Rewards for Weak-to-Strong Visual Captioning [57.588999592609646]
  We propose VCap, a Witness-Adjudicator reward that pairs the reference caption (a witness) with the visual signal (an adjudicator)<n>VCap delivers a reward signal with hypergeometric-distribution-level precision for caption quality verification.<n>In our experiments, an 8B model trained with VCap outperforms open- and closed-source SOTA models on multiple image and video captioning benchmarks.
  arXiv  Detail & Related papers  (2026-05-27T06:27:04Z)
• BalCapRL: A Balanced Framework for RL-Based MLLM Image Captioning [4.441850776689605]
  We propose a more balanced reinforcement learning framework that jointly optimize utility-aware correctness, reference coverage, and linguistic quality.<n>We show that our method consistently improves caption quality, with peak gains of +13.6 DCScore, +9.0 CaptionQA, and +29.0 Cap across different models.
  arXiv  Detail & Related papers  (2026-05-08T07:48:36Z)
• CCCaption: Dual-Reward Reinforcement Learning for Complete and Correct Image Captioning [23.289413412387223]
  We introduce CCCaption: a dual-reward reinforcement learning framework with a dedicated fine-tuning corpus.<n>For completeness, we use diverse LVLMs to disentangle the image into a set of visual queries, and reward captions that answer more of these queries.<n>For correctness, we penalize captions that contain hallucinations by validating the authenticity of sub-caption queries.
  arXiv  Detail & Related papers  (2026-02-25T07:34:26Z)
• CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning [90.19455861166745]
  We introduce Captioning Reinforcement Learning (CapRL), a training framework that redefines caption quality through its utility.<n>As the first study to apply RLVR to the subjective image captioning task, we demonstrate that CapRL significantly enhances multiple settings.<n>CapRL achieves performance comparable to Qwen2.5-VL-72B, while exceeding the baseline by an average margin of 8.4%.
  arXiv  Detail & Related papers  (2025-09-26T17:59:55Z)
• ScaleCap: Inference-Time Scalable Image Captioning via Dual-Modality Debiasing [128.8346376825612]
  Key challenges of high-quality image captioning lie in the inherent biases of LVLMs.<n>We propose a scalable debiased captioning strategy, which continuously enriches and calibrates the caption with increased inference budget.<n>Annotating 450K images with ScaleCap and using them for LVLM pretraining leads to consistent performance gains across 11 widely used benchmarks.
  arXiv  Detail & Related papers  (2025-06-24T17:59:55Z)
• Paraphrasing Is All You Need for Novel Object Captioning [126.66301869607656]
  Novel object captioning (NOC) aims to describe images containing objects without observing their ground truth captions during training. We present Paraphrasing-to-Captioning (P2C), a two-stage learning framework for NOC, which wouldally optimize the output captions via paraphrasing.
  arXiv  Detail & Related papers  (2022-09-25T22:56:04Z)
• Prompt-based Learning for Unpaired Image Captioning [86.44188293709307]
  Unpaired Image Captioning (UIC) has been developed to learn image descriptions from unaligned vision-language sample pairs. Recent successes of Vision-Language Pre-Trained Models (VL-PTMs) have triggered the development of prompt-based learning. We present in this paper a novel scheme based on prompt to train the UIC model, making best use of the powerful generalization ability.
  arXiv  Detail & Related papers  (2022-05-26T03:13:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.