TSR-Ego: Temporally Guided Stereo Refinement Framework for Egocentric 3D Human Pose Estimation

• URL: http://arxiv.org/abs/2607.09169v1
• Date: Fri, 10 Jul 2026 07:56:30 GMT
• Title: TSR-Ego: Temporally Guided Stereo Refinement Framework for Egocentric 3D Human Pose Estimation
• Abstract summary: Egocentric 3D human pose estimation from head-mounted stereo cameras is challenging due to fisheye distortion, severe self-occlusion, and frequent truncation of body joints.<n>We propose TSR-Ego, a temporally guided stereo framework that couples short-term motion evidence with projection-guided feature sampling.<n>Experiments on UnrealEgo2 and UnrealEgo-RW show state-of-the-art performance, with especially strong gains on real-world sequences.
• Score: 11.399976049695212
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Egocentric 3D human pose estimation from head-mounted stereo cameras is challenging due to fisheye distortion, severe self-occlusion, and frequent truncation of body joints outside the camera field of view. Recent stereo egocentric methods have improved performance through heatmap lifting, stereo correspondence, and transformer-based refinement, but they often rely heavily on frame-local evidence or use temporal information only as auxiliary pose-level context. This limits robustness when current-frame stereo cues are weak, occluded, or ambiguous. We propose TSR-Ego, a temporally guided stereo framework that couples short-term motion evidence with projection-guided feature sampling. The model first enriches dense stereo feature maps using a causal depthwise-separable temporal convolution, allowing past visual evidence to influence the feature space before deformable cross-attention. A single-stage causal stereo decoder then refines learned 3D joint queries through temporal self-attention, joint self-attention, and fisheye deformable stereo cross-attention, using the evolving pose estimate to generate 2D sampling references. Unlike methods that apply temporal reasoning mainly after pose prediction, TSR-Ego uses motion context to shape both the sampled stereo features and the joint representations while preserving online inference without future frames. Experiments on UnrealEgo2 and UnrealEgo-RW show state-of-the-art performance, with especially strong gains on real-world sequences.

Related papers

• EgoEV-HandPose: Egocentric 3D Hand Pose Estimation and Gesture Recognition with Stereo Event Cameras [27.55657128726312]
  EgoEV-HandPose is an end-to-end framework for joint 3D bimanual pose estimation and gesture recognition from stereo event streams.<n>EgoEVHands is the first large-scale real-world stereo event-camera dataset for egocentric hand perception.
  arXiv  Detail & Related papers  (2026-05-12T15:51:04Z)
• Bidirectional Cross-Modal Prompting for Event-Frame Asymmetric Stereo [54.58078274676216]
  Event cameras offer an alternative visual representation with higher dynamic range free of such limitations.<n>In this paper, we introduce Bi-CMPStereo, a novel bidirectional cross-modal prompting framework.<n>Our approach learns finely aligned stereo representations within a target canonical space and integrates complementary representations by projecting each modality into both event and frame domains.
  arXiv  Detail & Related papers  (2026-04-16T17:59:58Z)
• AG-EgoPose: Leveraging Action-Guided Motion and Kinematic Joint Encoding for Egocentric 3D Pose Estimation [11.399976049695212]
  AG-EgoPose is a novel dual-stream framework that integrates short- and long-range motion context with fine-grained spatial cues for robust pose estimation.<n>Our framework features two parallel streams: A spatial stream uses a weight-sharing ResNet-18 encoder-decoder to generate 2D joint heatmaps.<n>A temporal stream uses a ResNet-50 backbone to extract visual features, which are then processed by an action recognition backbone to capture the motion dynamics.
  arXiv  Detail & Related papers  (2026-03-26T08:46:51Z)
• Stereo World Model: Camera-Guided Stereo Video Generation [52.3922115596956]
  We present StereoWorld, a camera-conditioned stereo world model that jointly learns appearance and binocular geometry for end-to-end stereo video generation.<n>Unlike monocular RGB or RGBD approaches, StereoWorld operates exclusively within the RGB modality, while simultaneously grounding geometry directly from disparity.
  arXiv  Detail & Related papers  (2026-03-18T05:42:22Z)
• Controllable Egocentric Video Generation via Occlusion-Aware Sparse 3D Hand Joints [87.13154261503168]
  Motion-controllable video generation is crucial for egocentric applications in virtual reality and embodied AI.<n>Existing methods often struggle to achieve 3D-consistent fine-grained hand articulation.<n>We propose a novel framework that generates egocentric videos from a single reference frame.
  arXiv  Detail & Related papers  (2026-03-12T10:02:23Z)
• DreamPose3D: Hallucinative Diffusion with Prompt Learning for 3D Human Pose Estimation [18.240580699213197]
  We introduce DreamPose3D, a diffusion-based framework that combines action-aware reasoning with temporal imagination for 3D pose estimation.<n>DreamPose3D dynamically conditions the denoising process using task-relevant action prompts extracted from 2D pose sequences, capturing high-level intent.<n>To model the structural relationships between joints effectively, we introduce a representation encoder that incorporates kinematic joint affinity into the attention mechanism.<n>A hallucinative pose decoder predicts temporally coherent 3D pose during training, simulating how humans mentally reconstruct motion trajectories to resolve ambiguity in perception.
  arXiv  Detail & Related papers  (2025-11-12T17:31:41Z)
• Mono2Stereo: A Benchmark and Empirical Study for Stereo Conversion [88.67015254278859]
  We introduce the Mono2Stereo dataset, providing high-quality training data and benchmark to support in-depth exploration of stereo conversion.<n>We conduct an empirical study that yields two primary findings. 1) The differences between the left and right views are subtle, yet existing metrics consider overall pixels, failing to concentrate on regions critical to stereo effects.<n>We introduce a new evaluation metric, Stereo Intersection-over-Union, which harmonizes disparity and achieves a high correlation with human judgments on stereo effect.
  arXiv  Detail & Related papers  (2025-03-28T09:25:58Z)
• 3D Human Pose Perception from Egocentric Stereo Videos [67.9563319914377]
  We propose a new transformer-based framework to improve egocentric stereo 3D human pose estimation. Our method is able to accurately estimate human poses even in challenging scenarios, such as crouching and sitting. We will release UnrealEgo2, UnrealEgo-RW, and trained models on our project page.
  arXiv  Detail & Related papers  (2023-12-30T21:21:54Z)
• DynamicStereo: Consistent Dynamic Depth from Stereo Videos [91.1804971397608]
  We propose DynamicStereo to estimate disparity for stereo videos. The network learns to pool information from neighboring frames to improve the temporal consistency of its predictions. We also introduce Dynamic Replica, a new benchmark dataset containing synthetic videos of people and animals in scanned environments.
  arXiv  Detail & Related papers  (2023-05-03T17:40:49Z)
• Bridging Stereo Geometry and BEV Representation with Reliable Mutual Interaction for Semantic Scene Completion [45.171150395915056]
  3D semantic scene completion (SSC) is an ill-posed perception task that requires inferring a dense 3D scene from limited observations. Previous camera-based methods struggle to predict accurate semantic scenes due to inherent geometric ambiguity and incomplete observations. We resort to stereo matching technique and bird's-eye-view (BEV) representation learning to address such issues in SSC.
  arXiv  Detail & Related papers  (2023-03-24T12:33:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.