Incentivizing Temporal-Awareness in Egocentric Video Understanding Models

• URL: http://arxiv.org/abs/2603.27184v1
• Date: Sat, 28 Mar 2026 08:02:59 GMT
• Title: Incentivizing Temporal-Awareness in Egocentric Video Understanding Models
• Abstract summary: Multimodal large language models (MLLMs) have recently shown strong performance in visual understanding, yet they often lack temporal awareness.<n>This deficiency stems in part from training objectives that fail to explicitly reward temporal reasoning and instead rely on frame-level spatial shortcuts.<n>We propose Temporal Global Policy Optimization (TGPO), a reinforcement learning with verifiable rewards (RLVR) algorithm designed to incentivize temporal awareness in MLLMs.
• Score: 51.40541228498294
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal large language models (MLLMs) have recently shown strong performance in visual understanding, yet they often lack temporal awareness, particularly in egocentric settings where reasoning depends on the correct ordering and evolution of events. This deficiency stems in part from training objectives that fail to explicitly reward temporal reasoning and instead rely on frame-level spatial shortcuts. To address this limitation, we propose Temporal Global Policy Optimization (TGPO), a reinforcement learning with verifiable rewards (RLVR) algorithm designed to incentivize temporal awareness in MLLMs. TGPO contrasts model outputs generated from temporally ordered versus shuffled video frames to derive calibrated, globally normalized reward signals that explicitly favor temporally coherent reasoning. Integrated with GRPO and GSPO, TGPO supports cold-start RL training and effectively suppresses spatial shortcut behaviors learned by existing MLLMs. Experiments across five egocentric video benchmarks demonstrate that TGPO consistently improves temporal grounding and causal coherence, outperforming prior RL-based video reasoning approaches. Our results suggest that TGPO offers a simple and scalable pathway toward temporally robust MLLMs for egocentric video understanding.

Related papers

• TimeThink: Reasoning with Time for Video LLMs [28.549802513886046]
  We propose TimeThink, a reinforcement learning framework that guides temporal evidence discovery in Video-LLMs.<n>Our key idea is to treat temporal clue steps as the fundamental optimization primitive of video reasoning.<n>To enable scalable training, we construct TimeThink-RFT-20K, a dataset with automatically derived temporal evidence segments.
  arXiv  Detail & Related papers  (2026-07-06T13:50:15Z)
• Reasoning as Intersection: Consensus-Frame Alignment for Visual Focus in Video-MLLMs [81.04673240949074]
  Reinforcement learning has improved the reasoning ability of large language models.<n>Applying outcome-only rewards to video multimodal large language models provides limited guidance on which visual evidence should support the answer.<n>We introduce Consensus Frame GRPO, a temporal-annotation-free process-level reward framework for evidence-aware video reasoning.
  arXiv  Detail & Related papers  (2026-06-16T19:42:54Z)
• EvoVid: Temporal-Centric Self-Evolution for Video Large Language Models [44.97714162696323]
  Video Large Language Models (Video-LLMs) have demonstrated strong capabilities in video reasoning through reinforcement learning (RL)<n>We propose a temporal-centric self-evolving framework that enables Video-LLMs to improve directly from raw, unannotated videos.
  arXiv  Detail & Related papers  (2026-05-21T03:00:35Z)
• Reinforcing Structured Chain-of-Thought for Video Understanding [49.3086326382764]
  We introduce Summary-Driven Reinforcement Learning (SDRL)<n>SDRL is a single-stage RL framework that obviates the need forSupervised Fine-Tuning (SFT)<n>Our method achieves state-of-the-art performance on seven public VideoQA datasets.
  arXiv  Detail & Related papers  (2026-03-26T22:11:14Z)
• VideoZoomer: Reinforcement-Learned Temporal Focusing for Long Video Reasoning [49.35834435935727]
  VideoZoomer is a novel agentic framework that enables MLLMs to control their visual focus during reasoning.<n>Our 7B model delivers diverse and complex reasoning patterns, yielding strong performance across a broad set of long video understanding and reasoning benchmarks.<n>These emergent capabilities allow it to consistently surpass existing open-source models and even rival proprietary systems on challenging tasks.
  arXiv  Detail & Related papers  (2025-12-26T11:43:21Z)
• Reinforcement Learning Tuning for VideoLLMs: Reward Design and Data Efficiency [56.475612147721264]
  We propose a dual-reward formulation that supervises both semantic and temporal reasoning through discrete and continuous reward signals.<n>We evaluate our approach across eight representative video understanding tasks, including VideoQA, Temporal Video Grounding, and Grounded VideoQA.<n>Results underscore the importance of reward design and data selection in advancing reasoning-centric video understanding with MLLMs.
  arXiv  Detail & Related papers  (2025-06-02T17:28:26Z)
• MUSEG: Reinforcing Video Temporal Understanding via Timestamp-Aware Multi-Segment Grounding [55.32878803528196]
  Video temporal understanding is crucial for multimodal large language models (MLLMs) to reason over events in videos.<n>We propose MUSEG, a novel RL-based method that enhances temporal understanding by introducing timestamp-aware multi-segment grounding.<n>To facilitate effective learning, we design a customized RL training recipe with phased rewards that progressively guides the model toward temporally grounded reasoning.
  arXiv  Detail & Related papers  (2025-05-27T04:50:07Z)
• ViaRL: Adaptive Temporal Grounding via Visual Iterated Amplification Reinforcement Learning [68.76048244253582]
  We introduce ViaRL, the first framework to leverage rule-based reinforcement learning (RL) for optimizing frame selection in video understanding.<n>ViaRL utilizes the answer accuracy of a downstream model as a reward signal to train a frame selector through trial-and-error.<n>ViaRL consistently delivers superior temporal grounding performance and robust generalization across diverse video understanding tasks.
  arXiv  Detail & Related papers  (2025-05-21T12:29:40Z)
• Self-alignment of Large Video Language Models with Refined Regularized Preference Optimization [45.55180760002661]
  Large Video Language Models (LVLMs) struggle with fine-grained temporal understanding, hallucinate, and often make simple mistakes on even simple video question-answering tasks.<n>We propose a self-alignment framework that enables LVLMs to learn from their own errors.
  arXiv  Detail & Related papers  (2025-04-16T13:43:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.