From Perception to Planning: Evolving Ego-Centric Task-Oriented Spatiotemporal Reasoning via Curriculum Learning

• URL: http://arxiv.org/abs/2604.10517v1
• Date: Sun, 12 Apr 2026 08:14:49 GMT
• Title: From Perception to Planning: Evolving Ego-Centric Task-Oriented Spatiotemporal Reasoning via Curriculum Learning
• Abstract summary: We present EgoTSR, a curriculum-based framework for learning task-basedtemporal reasoning.<n>EgoTSR is built on the premise embodied reasoning should evolve from explicit spatial understanding to task-state assessment.<n>Experiments demonstrate that EgoTSR effectively eliminates biases, achieving 92.4% accuracy on long-horizon logical reasoning tasks.
• Score: 79.65573577667944
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Modern vision-language models achieve strong performance in static perception, but remain limited in the complex spatiotemporal reasoning required for embodied, egocentric tasks. A major source of failure is their reliance on temporal priors learned from passive video data, which often leads to spatiotemporal hallucinations and poor generalization in dynamic environments. To address this, we present EgoTSR, a curriculum-based framework for learning task-oriented spatiotemporal reasoning. EgoTSR is built on the premise that embodied reasoning should evolve from explicit spatial understanding to internalized task-state assessment and finally to long-horizon planning. To support this paradigm, we construct EgoTSR-Data, a large-scale dataset comprising 46 million samples organized into three stages: Chain-of-Thought (CoT) supervision, weakly supervised tagging, and long-horizon sequences. Extensive experiments demonstrate that EgoTSR effectively eliminates chronological biases, achieving 92.4% accuracy on long-horizon logical reasoning tasks while maintaining high fine-grained perceptual precision, significantly outperforming existing open-source and closed-source state-of-the-art models.

Related papers

• Lifelong Representations: A Survey on Continual Self-Supervised Learning for Vision Models [26.65040437876567]
  We present a comprehensive survey of continual self-supervised learning (CSSL) for vision.<n>We examine why self-supervised objectives exhibit improved robustness to catastrophic forgetting.<n>We identify open challenges such as scalability and the need for fast adaptability.
  arXiv  Detail & Related papers  (2026-07-08T13:44:58Z)
• PRTS: A Primitive Reasoning and Tasking System via Contrastive Representations [66.94988600664574]
  We present textbfPRTS (textbfPrimitive textbfReasoning and textbfTasking textbfSystem), a VLA foundation model that reformulates pretraining through Goal-Conditioned Reinforcement Learning.
  arXiv  Detail & Related papers  (2026-04-30T06:14:02Z)
• A Progressive Training Strategy for Vision-Language Models to Counteract Spatio-Temporal Hallucinations in Embodied Reasoning [49.61652671596548]
  "Multi-image hallucination reasoning", where a massive performance drop between forward and temporal queries reveals a dependence on superficial shortcuts instead of genuine understanding.<n>To mitigate this, we develop a new dataset: Chain-of-temporal building on detailed reasoning into Chain steps and definitive judgments.<n>Our experiments demonstrate that this approach not only improves accuracy but also the forward-backward performance gap from over 70% to only 6.53%.
  arXiv  Detail & Related papers  (2026-04-12T07:48:44Z)
• EXPLORE-Bench: Egocentric Scene Prediction with Long-Horizon Reasoning [63.010793398283134]
  We study whether multimodal large language models can reliably reason about the long-term physical consequences of actions from an egocentric viewpoint.<n>We introduce EXPLORE-Bench, a benchmark curated from real first-person videos spanning diverse scenarios.<n>Experiments on a range of proprietary and open-source MLLMs reveal a significant performance gap to humans.
  arXiv  Detail & Related papers  (2026-03-10T14:33:44Z)
• EgoReasoner: Learning Egocentric 4D Reasoning via Task-Adaptive Structured Thinking [34.29831801830779]
  EgoReasoner is a framework that aligns the reasoning scaffold and the reward signal to each task's cognitive structure.<n>Our model achieves 37.5% average accuracy on the challenging HD-EPIC benchmark, surpassing Qwen2.5-VL-7B by over 10 points.
  arXiv  Detail & Related papers  (2026-03-06T18:49:04Z)
• daVinci-Agency: Unlocking Long-Horizon Agency Data-Efficiently [35.39097522391409]
  Large Language Models (LLMs) excel at short-term tasks, scaling them to long-horizon agentic synthesis remains challenging.<n>We propose daVinci-Agency, which systematically mines structured supervision from chain-of-PRs.<n>DaVinci-Agency's PR-grounded structure inherently preserves the causal dependencies and iterative refinements essential for teaching persistent goal-cycle behavior.
  arXiv  Detail & Related papers  (2026-02-02T13:23:39Z)
• GSR: Learning Structured Reasoning for Embodied Manipulation [10.756456261056867]
  Grounded Scene-graph Reasoning (GSR) is a structured reasoning paradigm that explicitly models world-state evolution as transitions over semantically grounded scene graphs.<n>GSR enables explicit reasoning about action preconditions, consequences, and goal satisfaction in a physically grounded space.
  arXiv  Detail & Related papers  (2026-02-02T06:07:42Z)
• STReasoner: Empowering LLMs for Spatio-Temporal Reasoning in Time Series via Spatial-Aware Reinforcement Learning [16.11676643415448]
  patio-temporal reasoning in time series involves the explicit synthesis of temporal dynamics, spatial dependencies, and textual context.<n>This capability is vital for high-stakes decision-making in systems such as traffic networks, power grids, and disease propagation.<n>To address the gap, we introduce ST-Bench, a benchmark consisting of four core tasks, including etiological reasoning, entity identification, correlation reasoning, and in-context forecasting.<n>We then propose STReasoner, which empowers LLM to integrate time series, graph structure, and text for explicit reasoning.
  arXiv  Detail & Related papers  (2026-01-06T18:46:12Z)
• Provable Benefit of Curriculum in Transformer Tree-Reasoning Post-Training [76.12556589212666]
  We show that curriculum post-training avoids the exponential complexity bottleneck.<n>Under outcome-only reward signals, reinforcement learning finetuning achieves high accuracy with sample complexity.<n>We establish guarantees for test-time scaling, where curriculum-aware querying reduces both reward oracle calls and sampling cost from exponential to order.
  arXiv  Detail & Related papers  (2025-11-10T18:29:54Z)
• EgoThinker: Unveiling Egocentric Reasoning with Spatio-Temporal CoT [56.24624833924252]
  EgoThinker is a framework that endows MLs with robust egocentric reasoning capabilities through-temporal chain-of-thought supervision and a two-stage learning curriculum.<n>EgoThinker outperforms existing methods across multiple egocentric benchmarks, while achieving substantial improvements in fine-grained-temporal localization tasks.
  arXiv  Detail & Related papers  (2025-10-27T17:38:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.