Reinforcing Consistency in Video MLLMs with Structured Rewards

• URL: http://arxiv.org/abs/2604.01460v1
• Date: Wed, 01 Apr 2026 23:15:34 GMT
• Title: Reinforcing Consistency in Video MLLMs with Structured Rewards
• Abstract summary: Multimodal large language models (MLLMs) have achieved remarkable progress in video understanding.<n>We study this failure mode through a compositional consistency audit that decomposes a caption into supporting factual and temporal claims.<n>Our training objective integrates three complementary components: (1) an instance-aware scene-graph reward for factual objects, attributes, and relations; (2) a temporal reward for event ordering and repetition; and (3) a video-grounded VQA reward for hierarchical self-verification.
• Score: 14.560061824569333
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal large language models (MLLMs) have achieved remarkable progress in video understanding. However, seemingly plausible outputs often suffer from poor visual and temporal grounding: a model may fabricate object existence, assign incorrect attributes, or collapse repeated events while still producing a globally reasonable caption or answer. We study this failure mode through a compositional consistency audit that decomposes a caption into supporting factual and temporal claims, investigating whether a correct high-level prediction is actually backed by valid lower-level evidence. Our top-down audit reveals that even correct root relational claims often lack reliable attribute and existence support. This indicates that standard sentence-level supervision is a weak proxy for faithful video understanding. Furthermore, when turning to reinforcement learning (RL) for better alignment, standard sentence-level rewards often prove too coarse to accurately localize specific grounding failures. To address this, we replace generic sentence-level rewards with a structured reward built from factual and temporal units. Our training objective integrates three complementary components: (1) an instance-aware scene-graph reward for factual objects, attributes, and relations; (2) a temporal reward for event ordering and repetition; and (3) a video-grounded VQA reward for hierarchical self-verification. Across temporal, general video understanding, and hallucination-oriented benchmarks, this objective yields consistent gains on open-source backbones. These results suggest that structured reward shaping is a practical route to more faithful video understanding.

Related papers

• Claim-Level Rubric Rewards for Video Caption Reinforcement Learning [44.96572145476932]
  Claim-Level Rewards Rewards (CuRe) is a structured reward framework designed to address the reward-design bottleneck in reinforcement learning for dense video captioning.<n>CuRe reformulates reward modeling as fine-grained claim-level verification.
  arXiv  Detail & Related papers  (2026-07-06T14:33:41Z)
• TimeThink: Reasoning with Time for Video LLMs [28.549802513886046]
  We propose TimeThink, a reinforcement learning framework that guides temporal evidence discovery in Video-LLMs.<n>Our key idea is to treat temporal clue steps as the fundamental optimization primitive of video reasoning.<n>To enable scalable training, we construct TimeThink-RFT-20K, a dataset with automatically derived temporal evidence segments.
  arXiv  Detail & Related papers  (2026-07-06T13:50:15Z)
• SER: Learning to Ground Video Reasoning with Semantic Evidence Rewards [8.705689021755221]
  We propose Semantic Evidence Reward (SER) to reform RL as a constrained task.<n>Instead of computing pixel-level annotations, SER uses a referee enhancing a VLM to evaluate model-generated evidence claims.<n>On the VLM benchmark, SER achieves 49.6%, improving by 3.0 points over the strong evidence-ground baseline.
  arXiv  Detail & Related papers  (2026-06-23T15:50:33Z)
• BALTO: Balanced Token-Level Policy Optimization for Hallucination Mitigation [51.22170603236523]
  We propose a Balanced Token-level Policy Optimization framework for hallucination mitigation.<n>BalTO extracts checkable factual claims, verifies them against the reference context, and projects claim-level judgments to token-level labels.<n>Experiments on ConFiQA, RAGTruth, and FinLLM-Eval show that BALTO achieves the highest faithfulness across all six model--benchmark settings.
  arXiv  Detail & Related papers  (2026-06-14T16:25:59Z)
• Reasoning Resides in Layers: Restoring Temporal Reasoning in Video-Language Models with Layer-Selective Merging [62.24239343184817]
  Multimodal adaptation equips large language models with perceptual capabilities, but often weakens the reasoning ability inherited from language-only pretraining.<n>We propose MERIT, a training-free, task-driven model merging framework for restoring temporal reasoning in video-language models.
  arXiv  Detail & Related papers  (2026-04-13T12:41:50Z)
• STEAR: Layer-Aware Spatiotemporal Evidence Intervention for Hallucination Mitigation in Video Large Language Models [14.848157882117613]
  Large Video Language Models (Video-LLMs) are prone to hallucinations, often generating visually unsupported or incorrect temporal relations.<n>We propose STEAR, which identifies high-risk decoding steps and selects token-conditioned visual evidence from grounding-aware middle layers.<n>Experiments demonstrate that STEAR consistently reduces hallucinations while improving temporal consistency, faithfulness, and robustness.
  arXiv  Detail & Related papers  (2026-04-03T13:52:57Z)
• Incentivizing Temporal-Awareness in Egocentric Video Understanding Models [51.40541228498294]
  Multimodal large language models (MLLMs) have recently shown strong performance in visual understanding, yet they often lack temporal awareness.<n>This deficiency stems in part from training objectives that fail to explicitly reward temporal reasoning and instead rely on frame-level spatial shortcuts.<n>We propose Temporal Global Policy Optimization (TGPO), a reinforcement learning with verifiable rewards (RLVR) algorithm designed to incentivize temporal awareness in MLLMs.
  arXiv  Detail & Related papers  (2026-03-28T08:02:59Z)
• VideoTemp-o3: Harmonizing Temporal Grounding and Video Understanding in Agentic Thinking-with-Videos [44.23732277782877]
  In long-video understanding, uniform frame sampling often fails to capture key visual evidence, leading to degraded performance and increased hallucinations.<n>Recent agentic thinking-with-videos paradigms have emerged, adopting a localize-clip-answer pipeline.<n>We propose VideoTemp-o3, a unified agentic thinking-with-videos framework that jointly models video grounding and question answering.
  arXiv  Detail & Related papers  (2026-02-08T03:45:50Z)
• Temporally Consistent Object-Centric Learning by Contrasting Slots [23.203973564679508]
  We introduce a novel object-level temporal contrastive loss for video object-centric models.<n>Our method significantly improves the temporal consistency of the learned object-centric representations.
  arXiv  Detail & Related papers  (2024-12-18T19:46:04Z)
• On the Consistency of Video Large Language Models in Temporal Comprehension [57.985769348320616]
  Video large language models (Video-LLMs) can temporally ground language queries and retrieve video moments.<n>We conduct a study on prediction consistency -- a key indicator for robustness and trustworthiness of temporal grounding.
  arXiv  Detail & Related papers  (2024-11-20T00:47:17Z)
• Beyond Simple Sum of Delayed Rewards: Non-Markovian Reward Modeling for Reinforcement Learning [44.770495418026734]
  Reinforcement Learning (RL) empowers agents to acquire various skills by learning from reward signals. Traditional methods assume the existence of underlying Markovian rewards and that the observed delayed reward is simply the sum of instance-level rewards. We propose Composite Delayed Reward Transformer (CoDeTr), which incorporates a specialized in-sequence attention mechanism.
  arXiv  Detail & Related papers  (2024-10-26T13:12:27Z)
• Siamese Learning with Joint Alignment and Regression for Weakly-Supervised Video Paragraph Grounding [70.31050639330603]
  Video paragraph grounding aims at localizing multiple sentences with semantic relations and temporal order from an untrimmed video. Existing VPG approaches are heavily reliant on a considerable number of temporal labels that are laborious and time-consuming to acquire. We introduce and explore Weakly-Supervised Video paragraph Grounding (WSVPG) to eliminate the need of temporal annotations.
  arXiv  Detail & Related papers  (2024-03-18T04:30:31Z)
• Weakly-Supervised Video Object Grounding via Causal Intervention [82.68192973503119]
  We target at the task of weakly-supervised video object grounding (WSVOG), where only video-sentence annotations are available during model learning. It aims to localize objects described in the sentence to visual regions in the video, which is a fundamental capability needed in pattern analysis and machine learning.
  arXiv  Detail & Related papers  (2021-12-01T13:13:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.