RubiCap: Rubric-Guided Reinforcement Learning for Dense Image Captioning

• URL: http://arxiv.org/abs/2603.09160v1
• Date: Tue, 10 Mar 2026 03:51:27 GMT
• Title: RubiCap: Rubric-Guided Reinforcement Learning for Dense Image Captioning
• Abstract summary: RubiCap is a novelReinforcement learning framework that derives fine-grained, sample-specific reward signals from rubrics.<n> RubiCap achieves the highest win rates on CapArena, outperforming supervised distillation, prior RL methods, human-expert annotations, and GPT-4V-augmented outputs.
• Score: 15.881702574415861
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Dense image captioning is critical for cross-modal alignment in vision-language pretraining and text-to-image generation, but scaling expert-quality annotations is prohibitively expensive. While synthetic captioning via strong vision-language models (VLMs) is a practical alternative, supervised distillation often yields limited output diversity and weak generalization. Reinforcement learning (RL) could overcome these limitations, but its successes have so far been concentrated in verifiable domains that rely on deterministic checkers -- a luxury not available in open-ended captioning. We address this bottleneck with RubiCap, a novel RL framework that derives fine-grained, sample-specific reward signals from LLM-written rubrics. RubiCap first assembles a diverse committee of candidate captions, then employs an LLM rubric writer to extract consensus strengths and diagnose deficiencies in the current policy. These insights are converted into explicit evaluation criteria, enabling an LLM judge to decompose holistic quality assessment and replace coarse scalar rewards with structured, multi-faceted evaluations. Across extensive benchmarks, RubiCap achieves the highest win rates on CapArena, outperforming supervised distillation, prior RL methods, human-expert annotations, and GPT-4V-augmented outputs. On CaptionQA, it demonstrates superior word efficiency: our 7B model matches Qwen2.5-VL-32B-Instruct, and our 3B model surpasses its 7B counterpart. Remarkably, using the compact RubiCap-3B as a captioner produces stronger pretrained VLMs than those trained on captions from proprietary models.

Related papers

• Re$^3$Cap: Retrieval-Guided Refinement for Image Captioning Enhancement via Reinforcement Learning [13.794868838200465]
  We argue that multi-modal retrieval can serve as an effective reasoning signal for caption refinement.<n>We present the Retrieval-Guided Refinement for Image Captioning (Re$3$Cap), a retrieval-guided reasoning strategy that enhances image captioning without requiring additional annotations.
  arXiv  Detail & Related papers  (2026-08-21T17:07:41Z)
• CapRL++: Unified Reinforcement Learning with Verifiable Rewards for Dense Image and Video Captioning [86.9494763617273]
  We introduce Captioning Reinforcement Learning++ (CapRL++), a reference-free training framework that redefines caption quality through its utility.<n>We show that CapRL++ improves dense caption quality and strengthens caption-based pretraining across tasks such as spatial and temporal understanding.
  arXiv  Detail & Related papers  (2026-06-08T12:09:20Z)
• VCap: Hypergeometric Rewards for Weak-to-Strong Visual Captioning [57.588999592609646]
  We propose VCap, a Witness-Adjudicator reward that pairs the reference caption (a witness) with the visual signal (an adjudicator)<n>VCap delivers a reward signal with hypergeometric-distribution-level precision for caption quality verification.<n>In our experiments, an 8B model trained with VCap outperforms open- and closed-source SOTA models on multiple image and video captioning benchmarks.
  arXiv  Detail & Related papers  (2026-05-27T06:27:04Z)
• BalCapRL: A Balanced Framework for RL-Based MLLM Image Captioning [4.441850776689605]
  We propose a more balanced reinforcement learning framework that jointly optimize utility-aware correctness, reference coverage, and linguistic quality.<n>We show that our method consistently improves caption quality, with peak gains of +13.6 DCScore, +9.0 CaptionQA, and +29.0 Cap across different models.
  arXiv  Detail & Related papers  (2026-05-08T07:48:36Z)
• ITIScore: An Image-to-Text-to-Image Rating Framework for the Image Captioning Ability of MLLMs [84.09282931360089]
  ICBench is a large-scale image captioning benchmark that covers 12 content categories and consists of both short and long captions on 2K images.<n>We conduct extensive human subjective studies to obtain mean opinion scores (MOSs) across fine-grained evaluation dimensions.<n>We propose an automated evaluation metric, textbfITIScore, based on an image-to-text-to-image framework, which measures caption quality through reconstruction consistency.
  arXiv  Detail & Related papers  (2026-04-04T15:33:07Z)
• CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning [90.19455861166745]
  We introduce Captioning Reinforcement Learning (CapRL), a training framework that redefines caption quality through its utility.<n>As the first study to apply RLVR to the subjective image captioning task, we demonstrate that CapRL significantly enhances multiple settings.<n>CapRL achieves performance comparable to Qwen2.5-VL-72B, while exceeding the baseline by an average margin of 8.4%.
  arXiv  Detail & Related papers  (2025-09-26T17:59:55Z)
• ScaleCap: Inference-Time Scalable Image Captioning via Dual-Modality Debiasing [128.8346376825612]
  Key challenges of high-quality image captioning lie in the inherent biases of LVLMs.<n>We propose a scalable debiased captioning strategy, which continuously enriches and calibrates the caption with increased inference budget.<n>Annotating 450K images with ScaleCap and using them for LVLM pretraining leads to consistent performance gains across 11 widely used benchmarks.
  arXiv  Detail & Related papers  (2025-06-24T17:59:55Z)
• CapArena: Benchmarking and Analyzing Detailed Image Captioning in the LLM Era [41.135849912850695]
  We build a platform with over 6000 pairwise caption battles and high-quality human preference votes.<n>Our arena-style evaluation marks a milestone, showing that leading models like GPT-4o achieve or even surpass human performance.<n>We release CapArena-Auto, an accurate and efficient automated benchmark for detailed captioning, achieving 94.3% correlation with human rankings at just $4 per test.
  arXiv  Detail & Related papers  (2025-03-16T02:56:09Z)
• Multi-LLM Collaborative Caption Generation in Scientific Documents [30.856381292477177]
  We introduce a framework called Multi-LLM Collaborative Figure Caption Generation (MLBCAP)<n>Our approach unfolds in three key modules.<n>Human evaluations demonstrate that informative captions produced by our approach rank better than human-written captions.
  arXiv  Detail & Related papers  (2025-01-05T14:09:12Z)
• Toward Robust Hyper-Detailed Image Captioning: A Multiagent Approach and Dual Evaluation Metrics for Factuality and Coverage [50.84150600032693]
  Multimodal large language models (MLLMs) excel at generating highly detailed captions but often produce hallucinations.<n>We propose a multiagent approach that leverages LLM-MLLM collaboration to correct given captions.<n>Our proposed method significantly enhances the factual accuracy of captions, even improving those generated by GPT-4V.
  arXiv  Detail & Related papers  (2024-12-20T01:37:22Z)
• Exploring Annotation-free Image Captioning with Retrieval-augmented Pseudo Sentence Generation [21.54093527562344]
  We propose a new strategy where the prior knowledge from large pre-trained models (LPMs) is distilled and leveraged as supervision. Specifically, we introduce Retrieval-augmented Pseudo Sentence Generation (RaPSG), which can efficiently retrieve highly relevant short region descriptions. Experimental results indicate that our method outperforms SOTA captioning models across various settings.
  arXiv  Detail & Related papers  (2023-07-27T10:16:13Z)
• Prompt-based Learning for Unpaired Image Captioning [86.44188293709307]
  Unpaired Image Captioning (UIC) has been developed to learn image descriptions from unaligned vision-language sample pairs. Recent successes of Vision-Language Pre-Trained Models (VL-PTMs) have triggered the development of prompt-based learning. We present in this paper a novel scheme based on prompt to train the UIC model, making best use of the powerful generalization ability.
  arXiv  Detail & Related papers  (2022-05-26T03:13:43Z)
• Fine-grained Image Captioning with CLIP Reward [104.71533106301598]
  We propose using CLIP, a multimodal encoder trained on huge image-text pairs from web, to calculate multimodal similarity and use it as a reward function. We also propose a simple finetuning strategy of the CLIP text encoder to improve grammar that does not require extra text annotation. In experiments on text-to-image retrieval and FineCapEval, the proposed CLIP-guided model generates more distinctive captions than the CIDEr-optimized model.
  arXiv  Detail & Related papers  (2022-05-26T02:46:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.