Multi-Agent Self-Improving Reinforcement Learning for Video Reasoning

• URL: http://arxiv.org/abs/2608.28675v1
• Date: Tue, 25 Aug 2026 07:46:32 GMT
• Title: Multi-Agent Self-Improving Reinforcement Learning for Video Reasoning
• Abstract summary: Video reasoning tasks such as grounded video question answering and temporal grounding require selecting temporal evidence that supports the query.<n>We study whether a frozen verifier can also guide training.<n>Our framework couples a trainable emphGrounder with a frozen emphVerifier: the Grounder samples candidate trajectories and evidence segments.
• Score: 45.361080098317665
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video reasoning tasks such as grounded video question answering and temporal grounding require selecting temporal evidence that supports the query. In many current training setups, temporal supervision is applied through local objectives such as boundary regression or span generation, while verification is used mainly to rerank candidate segments at inference time. We study whether a frozen verifier can also guide training. Our multi-agent framework couples a trainable \emph{Grounder} with a frozen \emph{Verifier}: the Grounder samples candidate trajectories and evidence segments, the Verifier assigns query-conditioned segment scores, a group-relative policy-gradient objective favors trajectories that outperform their within-input peers, and a bootstrapped calibration loss steers temporal predictions toward verifier-preferred spans. Trained on source tasks and evaluated without target-dataset fine-tuning, a two-billion-parameter instantiation transfers zero-shot across grounded question answering, temporal grounding, and long-video question answering, reaching 28.7\% intersection-over-union and 25.4\% answer-grounding accuracy on a grounded-question-answering benchmark, 46.1\% intersection-over-union on a temporal-grounding benchmark, and 54.1\% on a long-video question-answering benchmark. Relative to a strong same-scale baseline, the gains are modest but consistent, with the clearest improvements on relevance-oriented metrics such as intersection-over-union and moderate-overlap recall. Within the tested benchmarks and transfer setting, the results support frozen verification as a training signal for evidence selection, while showing that strict boundary precision remains comparatively weaker. Code and models are available at https://anonymous.4open.science/r/MASIRL-E50C/

Related papers

• Concentrate After Imagination: Text-Conditioned Evidence Grounding for Partially Relevant Video Retrieval [27.64333612353748]
  Partially Relevant Video Retrieval retrieves untrimmed videos when queries describe only short moments.<n>We propose TRACE, a score-level evidence verification operator for PRVR.<n>On ActivityNet Captions, Charades-STA, and TVR, TRACE achieves the best SumR on all three benchmarks.
  arXiv  Detail & Related papers  (2026-09-08T16:37:35Z)
• Disagree to Explore, Agree to Commit: Routing-Guided Test-Time Scaling for Software Agents [60.650808962786364]
  We study whether native MoE router traces can guide steering and selection without an external judge or selection-time test execution.<n>Our analysis shows that routing provides a robust behavioral role signal; token-granular readouts and decision-matched comparison sets turn it into effective control.<n>We evaluate on SWE-bench Verified using open-weight sparse MoE agents across scales and reasoning-effort settings.
  arXiv  Detail & Related papers  (2026-08-23T03:07:03Z)
• Beyond Perplexity: A Behavioral Evaluation Framework for Deployment-Memory Claims in LLM Test-Time Training [30.22002300048916]
  Large language model test-time training (TTT) is often evaluated through local proxy metrics.<n>We introduce a behavioral evaluation framework that calibrates TTT memory claims to the evidence that supports them.
  arXiv  Detail & Related papers  (2026-07-01T03:07:17Z)
• Affordance Agent Harness: Verification-Gated Skill Orchestration [45.231685718099264]
  Affordance grounding requires identifying where and how an agent should interact in open-world scenes.<n>We propose Affordance Agent Harness, a closed-loop runtime that unifies skills with an evidence store and cost control.
  arXiv  Detail & Related papers  (2026-05-01T13:45:16Z)
• Find, Fix, Reason: Context Repair for Video Reasoning [45.021693494492666]
  Reinforcement learning has advanced video reasoning in large multi-modal models.<n>We introduce an observation-level intervention: a frozen, tool-integrated teacher identifies the missingtemporal dependency and provides a minimal evidence patch.<n>We propose a Robust Improvement Reward (RIR) that aligns optimization with two goals: outcome validity through correct answers and dependency alignment rationales.
  arXiv  Detail & Related papers  (2026-04-17T17:04:19Z)
• VideoZeroBench: Probing the Limits of Video MLLMs with Spatio-Temporal Evidence Verification [73.02304272829785]
  VideoBench is a hierarchical benchmark for long-video answering that rigorously verifies evidence.<n>It comprises 500 manually annotated questions across 13 domains, paired with temporal intervals and spatial bounding boxes as evidence.<n>Experiments show that even GeminiPro-3-Pro correctly answers fewer than 17% of questions under the standard end-to-end QA setting.<n>Results expose a significant gap between surface-level answer correctness and genuine evidence-based reasoning.
  arXiv  Detail & Related papers  (2026-04-02T03:29:43Z)
• Noisy Correspondence Learning with Self-Reinforcing Errors Mitigation [63.180725016463974]
  Cross-modal retrieval relies on well-matched large-scale datasets that are laborious in practice. We introduce a novel noisy correspondence learning framework, namely textbfSelf-textbfReinforcing textbfErrors textbfMitigation (SREM)
  arXiv  Detail & Related papers  (2023-12-27T09:03:43Z)
• A Closer Look at Debiased Temporal Sentence Grounding in Videos: Dataset, Metric, and Approach [53.727460222955266]
  Temporal Sentence Grounding in Videos (TSGV) aims to ground a natural language sentence in an untrimmed video. Recent studies have found that current benchmark datasets may have obvious moment annotation biases. We introduce a new evaluation metric "dR@n,IoU@m" that discounts the basic recall scores to alleviate the inflating evaluation caused by biased datasets.
  arXiv  Detail & Related papers  (2022-03-10T08:58:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.