Rigel: Self-Distilled Score Adaptation for Image and Video Captioning Evaluation

• URL: http://arxiv.org/abs/2606.29997v1
• Date: Mon, 29 Jun 2026 09:07:29 GMT
• Title: Rigel: Self-Distilled Score Adaptation for Image and Video Captioning Evaluation
• Abstract summary: Rigel is an automatic evaluation metric for image and video captioning based on self-distilled score adaptation.<n>To train Rigel, we constructed the Vid-Lepus dataset, which contains 3,338 video clips, 33,380 reference captions, and 5,637 candidate captions.
• Score: 2.964340015030302
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automatic evaluation of image and video captioning is essential for benchmarking multimodal systems, although standard evaluation metrics show limited alignment with human judgments. Recent approaches using large language models (LLMs), commonly referred to as LLM-as-a-Judge, have improved alignment with human judgments but still suffer from a mismatch between large-vocabulary language modeling and evaluation over a small label set. To address this, we propose Rigel, an automatic evaluation metric for image and video captioning, based on self-distilled score adaptation. The metric employs an evaluation-specific scoring head distilled from a frozen LLM, which captures judgment signals in a task-aligned space without relying on large-vocabulary token sets. We then refine the LLM backbone with human judgment data. To train Rigel, we constructed the Vid-Lepus dataset, which contains 3,338 video clips, 33,380 reference captions, and 5,637 candidate captions. Experiments on multiple benchmarks show that Rigel outperforms state-of-the-art metrics, achieving over 10-point improvements on ActivityNet-Fact in the reference-free setting.

Related papers

• ITIScore: An Image-to-Text-to-Image Rating Framework for the Image Captioning Ability of MLLMs [84.09282931360089]
  ICBench is a large-scale image captioning benchmark that covers 12 content categories and consists of both short and long captions on 2K images.<n>We conduct extensive human subjective studies to obtain mean opinion scores (MOSs) across fine-grained evaluation dimensions.<n>We propose an automated evaluation metric, textbfITIScore, based on an image-to-text-to-image framework, which measures caption quality through reconstruction consistency.
  arXiv  Detail & Related papers  (2026-04-04T15:33:07Z)
• VELA: An LLM-Hybrid-as-a-Judge Approach for Evaluating Long Image Captions [3.8028282626618526]
  VELA is an automatic evaluation metric for long captions developed within a novel LLM-Hybrid-as-a-Judge framework.<n>LongCap-Arena is a benchmark specifically designed for evaluating metrics for long captions.
  arXiv  Detail & Related papers  (2025-09-30T05:52:34Z)
• VideoJudge: Bootstrapping Enables Scalable Supervision of MLLM-as-a-Judge for Video Understanding [57.15309719147799]
  We introduce VideoJudge, a 3B and 7B-sized MLLM judge specialized to evaluate outputs from video understanding models.<n>To train VideoJudge, our recipe builds on the interplay between a generator and an evaluator.<n>Across three out of four meta-evaluation benchmarks, VideoJudge-7B outperforms larger MLLM judge baselines.
  arXiv  Detail & Related papers  (2025-09-25T19:22:57Z)
• Painting with Words: Elevating Detailed Image Captioning with Benchmark and Alignment Learning [56.31096024472269]
  We introduce DeCapBench along with a novel metric, DCScore, specifically designed for detailed captioning tasks.<n>DCScore evaluates hallucinations and fine-grained comprehensiveness by deconstructing responses into the smallest self-sufficient units.<n>DeCapBench exhibits a high correlation with VLM arena results on descriptive tasks, surpassing existing benchmarks for vision-language models.
  arXiv  Detail & Related papers  (2025-03-10T22:53:56Z)
• Evaluating Image Caption via Cycle-consistent Text-to-Image Generation [24.455344211552692]
  We propose CAMScore, a reference-free automatic evaluation metric for image captioning models.<n>To circumvent the aforementioned modality gap, CAMScore utilizes a text-to-image model to generate images from captions and subsequently evaluates these generated images against the original images.<n>Experiment results show that CAMScore achieves a superior correlation with human judgments compared to existing reference-based and reference-free metrics.
  arXiv  Detail & Related papers  (2025-01-07T06:35:34Z)
• Toward Robust Hyper-Detailed Image Captioning: A Multiagent Approach and Dual Evaluation Metrics for Factuality and Coverage [50.84150600032693]
  Multimodal large language models (MLLMs) excel at generating highly detailed captions but often produce hallucinations.<n>We propose a multiagent approach that leverages LLM-MLLM collaboration to correct given captions.<n>Our proposed method significantly enhances the factual accuracy of captions, even improving those generated by GPT-4V.
  arXiv  Detail & Related papers  (2024-12-20T01:37:22Z)
• FineSurE: Fine-grained Summarization Evaluation using LLMs [22.62504593575933]
  FineSurE is a fine-grained evaluator specifically tailored for the summarization task using large language models (LLMs) It also employs completeness and conciseness criteria, in addition to faithfulness, enabling multi-dimensional assessment.
  arXiv  Detail & Related papers  (2024-07-01T02:20:28Z)
• Using Natural Language Explanations to Rescale Human Judgments [81.66697572357477]
  We propose a method to rescale ordinal annotations and explanations using large language models (LLMs)<n>We feed annotators' Likert ratings and corresponding explanations into an LLM and prompt it to produce a numeric score anchored in a scoring rubric.<n>Our method rescales the raw judgments without impacting agreement and brings the scores closer to human judgments grounded in the same scoring rubric.
  arXiv  Detail & Related papers  (2023-05-24T06:19:14Z)
• EMScore: Evaluating Video Captioning via Coarse-Grained and Fine-Grained Embedding Matching [90.98122161162644]
  Current metrics for video captioning are mostly based on the text-level comparison between reference and candidate captions. We propose EMScore (Embedding Matching-based score), a novel reference-free metric for video captioning. We exploit a well pre-trained vision-language model to extract visual and linguistic embeddings for computing EMScore.
  arXiv  Detail & Related papers  (2021-11-17T06:02:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.