EgoReasoner: Learning Egocentric 4D Reasoning via Task-Adaptive Structured Thinking

• URL: http://arxiv.org/abs/2603.06561v1
• Date: Fri, 06 Mar 2026 18:49:04 GMT
• Title: EgoReasoner: Learning Egocentric 4D Reasoning via Task-Adaptive Structured Thinking
• Abstract summary: EgoReasoner is a framework that aligns the reasoning scaffold and the reward signal to each task's cognitive structure.<n>Our model achieves 37.5% average accuracy on the challenging HD-EPIC benchmark, surpassing Qwen2.5-VL-7B by over 10 points.
• Score: 34.29831801830779
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Egocentric video understanding is inherently complex due to the dynamic 4D nature of the environment, where camera motion and object displacements necessitate a continuous re-evaluation of spatial relations. In this work, we target a suite of under-explored egocentric 4D reasoning tasks, including fixture interaction counting, viewpoint-relative fixture location, object movement itinerary tracking, and stationary object localization, that require fundamentally different cognitive operations: spatial anchoring, temporal tracking, and duration reasoning. We observe that these structural differences make task-agnostic approaches insufficient: generic Chain-of-Thought methods lack task-appropriate reasoning primitives, and uniform reinforcement learning actively destabilizes performance on spatial tasks. To address this, we propose EgoReasoner, a two-stage framework that aligns both the reasoning scaffold and the reward signal to each task's cognitive structure. In the first stage, Task-Adaptive Thinking Templates guide the synthesis of structured CoT traces that teach the model to reason adaptively across task types via supervised fine-tuning. In the second stage, task-aware reward functions verify entity grounding, temporal alignment, and task-adaptive logical consistency, selectively strengthening each reasoning pathway via reinforcement fine-tuning with GRPO. Our 3B-parameter model, trained on only 16K samples, achieves 37.5% average accuracy on the challenging HD-EPIC benchmark, surpassing Qwen2.5-VL-7B (25.7%) by over 10 points.

Related papers

• Spatial-IQ: Deconstructing Spatial Intelligence via Hierarchical Capability Tests [8.148716256952607]
  Multimodal large language models (MLLMs) excel at visual interpretation but fail on spatial reasoning tasks that humans solve reliably.<n>We introduce Spatial-IQ, a hierarchical diagnostic framework that decomposes object counting in stacked 3D structures into 9 perceptual and cognitive sub-tasks.<n>We evaluate models across three output formats (free-response text, multiple-choice images, and image editing) alongside a human baseline.
  arXiv  Detail & Related papers  (2026-07-24T19:09:25Z)
• Self in Space: Benchmarking Self-Awareness and Spatial Cognition in UAV Embodied Intelligence [23.612425637328553]
  SIS-Bench is a benchmark for evaluating embodied spatial intelligence in UAV scenarios.<n> SIS-Bench organizes evaluation along two complementary dimensions, space and self, and a three-level hierarchy of perception, memory, and reasoning.<n> Experimental results show that modeling agent motion consistently improves perception and memory performance.
  arXiv  Detail & Related papers  (2026-07-14T08:04:31Z)
• The Art of Interrogation: Consistency Amplifies Factuality in Spatial Reasoning [65.67266333751569]
  Current Large Reasoning Models (LRMs) exhibit remarkable general capabilities but significantly underperform in spatial reasoning tasks.<n>We propose a self-supervised reinforcement learning framework that targets the internal reasoning process without requiring ground-truth annotations.<n>We show that this label-free consistency training approaches the accuracy of models trained with ground-truth supervision and achieves similar generalization across diverse tasks and data domains.
  arXiv  Detail & Related papers  (2026-06-10T10:50:06Z)
• Embodied3DBench: Benchmarking Low-Level Embodied Spatial Intelligence of Vision Language Models [50.6415287154632]
  Embodied3DBench is a robot-centric benchmark targeting low-level spatial intelligence in 3D environments.<n>The benchmark spans 12 subcategories and contains over 21k high-quality question-answer pairs.
  arXiv  Detail & Related papers  (2026-05-27T20:28:56Z)
• From Perception to Planning: Evolving Ego-Centric Task-Oriented Spatiotemporal Reasoning via Curriculum Learning [79.65573577667944]
  We present EgoTSR, a curriculum-based framework for learning task-basedtemporal reasoning.<n>EgoTSR is built on the premise embodied reasoning should evolve from explicit spatial understanding to task-state assessment.<n>Experiments demonstrate that EgoTSR effectively eliminates biases, achieving 92.4% accuracy on long-horizon logical reasoning tasks.
  arXiv  Detail & Related papers  (2026-04-12T08:14:49Z)
• From Perception to Action: An Interactive Benchmark for Vision Reasoning [51.11355591375073]
  Causal Hierarchy of Actions and Interactions (CHAIN) benchmark designed to evaluate whether models can understand, plan, and execute structured action sequences grounded in physical constraints.<n> CHAIN shifts evaluation from passive perception to active problem solving, spanning tasks such as interlocking mechanical puzzles and 3D stacking and packing.<n>Our results show that top-performing models still struggle to internalize physical structure and causal constraints, often failing to produce reliable long-horizon plans and cannot robustly translate perceived structure into effective actions.
  arXiv  Detail & Related papers  (2026-02-24T15:33:02Z)
• Understanding Dynamic Scenes in Ego Centric 4D Point Clouds [7.004204907286336]
  EgoDynamic4D is a novel QA benchmark on highly dynamic scenes.<n>We design 12 dynamic QA tasks covering agent motion, human-object interaction prediction, relation, understanding trajectory, temporal-causal reasoning, with fine-grained, explicit metrics.<n>Our method consistently outperforms baselines, validating the effectiveness of multimodal temporal modeling for ego-centric dynamic scene understanding.
  arXiv  Detail & Related papers  (2025-08-10T09:08:04Z)
• SpatialCoT: Advancing Spatial Reasoning through Coordinate Alignment and Chain-of-Thought for Embodied Task Planning [42.487500113839666]
  We propose a novel approach to bolster the spatial reasoning capabilities of Vision-Language Models (VLMs)<n>Our approach comprises two stages: spatial coordinate bi-directional alignment, and chain-of-thought spatial grounding.<n>We evaluate our method on challenging navigation and manipulation tasks, both in simulation and real-world settings.
  arXiv  Detail & Related papers  (2025-01-17T09:46:27Z)
• Towards Robust and Realistic Human Pose Estimation via WiFi Signals [85.60557095666934]
  WiFi-based human pose estimation is a challenging task that bridges discrete and subtle WiFi signals to human skeletons.<n>This paper revisits this problem and reveals two critical yet overlooked issues: 1) cross-domain gap, i.e., due to significant variations between source-target domain pose distributions; and 2) structural fidelity gap, i.e., predicted skeletal poses manifest distorted topology.<n>This paper fills these gaps by reformulating the task into a novel two-phase framework dubbed DT-Pose: Domain-consistent representation learning and Topology-constrained Pose decoding.
  arXiv  Detail & Related papers  (2025-01-16T09:38:22Z)
• Grounding 3D Scene Affordance From Egocentric Interactions [52.5827242925951]
  Grounding 3D scene affordance aims to locate interactive regions in 3D environments. We introduce a novel task: grounding 3D scene affordance from egocentric interactions.
  arXiv  Detail & Related papers  (2024-09-29T10:46:19Z)
• Localizing Active Objects from Egocentric Vision with Symbolic World Knowledge [62.981429762309226]
  The ability to actively ground task instructions from an egocentric view is crucial for AI agents to accomplish tasks or assist humans virtually. We propose to improve phrase grounding models' ability on localizing the active objects by: learning the role of objects undergoing change and extracting them accurately from the instructions. We evaluate our framework on Ego4D and Epic-Kitchens datasets.
  arXiv  Detail & Related papers  (2023-10-23T16:14:05Z)
• Reinforcement Learning for Sparse-Reward Object-Interaction Tasks in a First-person Simulated 3D Environment [73.9469267445146]
  First-person object-interaction tasks in high-fidelity, 3D, simulated environments such as the AI2Thor pose significant sample-efficiency challenges for reinforcement learning agents. We show that one can learn object-interaction tasks from scratch without supervision by learning an attentive object-model as an auxiliary task.
  arXiv  Detail & Related papers  (2020-10-28T19:27:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.