EgoExo-Con: Exploring View-Invariant Video Temporal Understanding

• URL: http://arxiv.org/abs/2510.26113v1
• Date: Thu, 30 Oct 2025 03:53:22 GMT
• Title: EgoExo-Con: Exploring View-Invariant Video Temporal Understanding
• Abstract summary: Can Video-LLMs achieve consistent temporal understanding when videos capture the same event from different viewpoints?<n>EgoExo-Con (Consistency) is a benchmark of comprehensively synchronized egocentric and exocentric video pairs with human-refined queries in natural language.<n>We propose View-GRPO, a novel reinforcement learning framework that effectively strengthens view-specific temporal reasoning.
• Score: 66.25513481642845
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Can Video-LLMs achieve consistent temporal understanding when videos capture the same event from different viewpoints? To study this, we introduce EgoExo-Con (Consistency), a benchmark of comprehensively synchronized egocentric and exocentric video pairs with human-refined queries in natural language. EgoExo-Con emphasizes two temporal understanding tasks: Temporal Verification and Temporal Grounding. It evaluates not only correctness but consistency across viewpoints. Our analysis reveals two critical limitations of existing Video-LLMs: (1) models often fail to maintain consistency, with results far worse than their single-view performances. (2) When naively finetuned with synchronized videos of both viewpoints, the models show improved consistency but often underperform those trained on a single view. For improvements, we propose View-GRPO, a novel reinforcement learning framework that effectively strengthens view-specific temporal reasoning while encouraging consistent comprehension across viewpoints. Our method demonstrates its superiority over naive SFT and GRPO, especially for improving cross-view consistency. All resources will be made publicly available.

Related papers

• Moving Beyond More Views: Redundancy-Aware Ego-Exo Fusion for Proficiency Estimation [12.151605559533337]
  EgoExo estimation aims to integrate fine-grained motion cues from egocentric (1st-person) views with spatial context from multiple exocentric (3rd-person) views.<n>Our analysis identifies two key causes: (1) Multiview redundancy - From the data perspective, certain views provide limited or noisy information, diluting discriminative cues; (2) Overfitting - From the feature perspective, conventional fusion increases representational complexity, causing the model to memorise view-specific patterns rather than learn generalisable representations.
  arXiv  Detail & Related papers  (2026-08-26T12:47:42Z)
• EvoVid: Temporal-Centric Self-Evolution for Video Large Language Models [44.97714162696323]
  Video Large Language Models (Video-LLMs) have demonstrated strong capabilities in video reasoning through reinforcement learning (RL)<n>We propose a temporal-centric self-evolving framework that enables Video-LLMs to improve directly from raw, unannotated videos.
  arXiv  Detail & Related papers  (2026-05-21T03:00:35Z)
• EgoCoT-Bench: Benchmarking Grounded and Verifiable Operation-Centric Chain of Thought Reasoning for MLLMs [23.973683864983272]
  EgoT-Bench is a fine-grained egocentric benchmark for grounded and verifiable operation-centric reasoning.<n>EgoT-Bench comprises 3,172 QA pairs over 351 egocentric retrospect videos.
  arXiv  Detail & Related papers  (2026-05-19T09:02:20Z)
• Video-MME-v2: Towards the Next Stage in Benchmarks for Comprehensive Video Understanding [98.3098451637867]
  Video-MME-v2 is a comprehensive benchmark designed to rigorously evaluate the robustness and faithfulness of video understanding.<n>To guarantee data quality, Video-MME-v2 is constructed through a rigorously controlled human annotation pipeline.
  arXiv  Detail & Related papers  (2026-04-06T17:59:56Z)
• VisionCoach: Reinforcing Grounded Video Reasoning via Visual-Perception Prompting [58.508924874097715]
  VisonCoach consists of two components: (1) Visual Prompt Selector, which predicts appropriate prompt types conditioned on the video and question, and (2) Spatio-Reasoner, optimized with RL under visual prompt guidance and object grounding rewards.<n>Our results show that visual prompting during training improves grounded video reasoning, while distract-distillationSense enables the model to this ability without requiring prompts at inference time.
  arXiv  Detail & Related papers  (2026-03-15T23:32:02Z)
• Thinking in Frames: How Visual Context and Test-Time Scaling Empower Video Reasoning [38.651924340946785]
  We formulate visual reasoning by means of video generation models.<n>We evaluate their capacity in two distinct regimes: Maze Navigation for sequential discrete planning with low visual change and Tangram Puzzle for continuous manipulation with high visual change.
  arXiv  Detail & Related papers  (2026-01-28T20:57:55Z)
• STARCaster: Spatio-Temporal AutoRegressive Video Diffusion for Identity- and View-Aware Talking Portraits [44.82339975771063]
  STARCaster is an identity-aware video diffusion model that addresses both speech-driven portrait animation and free-point-view talking portrait.<n>The model learns from longer temporal contexts than those generated at inference, mitigating the overly static animations common in existing autoregressive approaches.
  arXiv  Detail & Related papers  (2025-12-15T11:59:01Z)
• Video-R2: Reinforcing Consistent and Grounded Reasoning in Multimodal Language Models [56.851611990473174]
  Reasoning over dynamic visual content remains a central challenge for large language models.<n>We propose a reinforcement learning approach that enhances both temporal precision and reasoning consistency.<n>The resulting model, Video R2, achieves consistently higher TAC, VAS, and accuracy across multiple benchmarks.
  arXiv  Detail & Related papers  (2025-11-28T18:59:58Z)
• Video-LLMs with Temporal Visual Screening [59.18455762289321]
  Temporal Visual Screening (TVS) is a new task that universally pre-processes video question answering and instruction tuning data.<n>TVS is formulated as a modular front-end adapter task that can be seamlessly integrated into both Video Instruction Tuning (training) and Video Question Answering (inference) pipelines.<n> Experiments demonstrate that incorporating TVS yields relative gains of 7.33% (training) and 34.6% (inference)
  arXiv  Detail & Related papers  (2025-08-27T14:33:32Z)
• STEP: Enhancing Video-LLMs' Compositional Reasoning by Spatio-Temporal Graph-guided Self-Training [87.58996020705258]
  Video Large Language Models (Video-LLMs) have recently shown strong derivation in basic video understanding tasks.<n>Video-LLMs struggle with compositional reasoning that requires multi-step explicit-temporal inference across object relations, interactions and events.<n>We propose STEP, a novel graph-guided self-training method that enables VideoLLMs to generate reasoning-rich finetuning data from any raw videos to improve itself.
  arXiv  Detail & Related papers  (2024-11-29T11:54:55Z)
• TOMATO: Assessing Visual Temporal Reasoning Capabilities in Multimodal Foundation Models [55.48403691519395]
  TOMATO is a novel benchmark crafted to rigorously assess MFMs' temporal reasoning capabilities in video understanding.<n>TOMATO comprises 1,484 carefully curated, human-annotated questions spanning six tasks.<n>Our comprehensive evaluation reveals a human-model performance gap of 57.3% with the best-performing model.
  arXiv  Detail & Related papers  (2024-10-30T17:50:23Z)
• Do Egocentric Video-Language Models Truly Understand Hand-Object Interactions? [48.702973928321946]
  Egocentric video-language pretraining is a crucial step in advancing the understanding of hand-object interactions in first-person scenarios.<n>Despite successes on existing testbeds, we find that current EgoVLMs can be easily misled by simple modifications.<n>This raises the question: Do EgoVLMs truly understand hand-object interactions?
  arXiv  Detail & Related papers  (2024-05-28T00:27:29Z)
• No More Shortcuts: Realizing the Potential of Temporal Self-Supervision [69.59938105887538]
  We propose a more challenging reformulation of temporal self-supervision as frame-level (rather than clip-level) recognition tasks. We demonstrate experimentally that our more challenging frame-level task formulations and the removal of shortcuts drastically improve the quality of features learned through temporal self-supervision.
  arXiv  Detail & Related papers  (2023-12-20T13:20:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.