E-3DPSM: A State Machine for Event-Based Egocentric 3D Human Pose Estimation

• URL: http://arxiv.org/abs/2604.08543v1
• Date: Thu, 09 Apr 2026 17:59:52 GMT
• Title: E-3DPSM: A State Machine for Event-Based Egocentric 3D Human Pose Estimation
• Abstract summary: Event cameras offer advantages in monocular egocentric 3D human pose estimation from head-mounted devices.<n> Existing methods suffer from low 3D estimation accuracy, insufficient in many applications.<n>This paper introduces E-3DPSM, an event-driven continuous pose state machine for event-based egocentric 3D human pose estimation.
• Score: 73.63705002652638
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Event cameras offer multiple advantages in monocular egocentric 3D human pose estimation from head-mounted devices, such as millisecond temporal resolution, high dynamic range, and negligible motion blur. Existing methods effectively leverage these properties, but suffer from low 3D estimation accuracy, insufficient in many applications (e.g., immersive VR/AR). This is due to the design not being fully tailored towards event streams (e.g., their asynchronous and continuous nature), leading to high sensitivity to self-occlusions and temporal jitter in the estimates. This paper rethinks the setting and introduces E-3DPSM, an event-driven continuous pose state machine for event-based egocentric 3D human pose estimation. E-3DPSM aligns continuous human motion with fine-grained event dynamics; it evolves latent states and predicts continuous changes in 3D joint positions associated with observed events, which are fused with direct 3D human pose predictions, leading to stable and drift-free final 3D pose reconstructions. E-3DPSM runs in real-time at 80 Hz on a single workstation and sets a new state of the art in experiments on two benchmarks, improving accuracy by up to 19% (MPJPE) and temporal stability by up to 2.7x. See our project page for the source code and trained models.

Related papers

• DreamPose3D: Hallucinative Diffusion with Prompt Learning for 3D Human Pose Estimation [18.240580699213197]
  We introduce DreamPose3D, a diffusion-based framework that combines action-aware reasoning with temporal imagination for 3D pose estimation.<n>DreamPose3D dynamically conditions the denoising process using task-relevant action prompts extracted from 2D pose sequences, capturing high-level intent.<n>To model the structural relationships between joints effectively, we introduce a representation encoder that incorporates kinematic joint affinity into the attention mechanism.<n>A hallucinative pose decoder predicts temporally coherent 3D pose during training, simulating how humans mentally reconstruct motion trajectories to resolve ambiguity in perception.
  arXiv  Detail & Related papers  (2025-11-12T17:31:41Z)
• StarPose: 3D Human Pose Estimation via Spatial-Temporal Autoregressive Diffusion [29.682018018059043]
  StarPose is an autoregressive diffusion framework for 3D human pose estimation.<n>It incorporates historical 3D pose predictions and spatial-temporal physical guidance.<n>It achieves superior accuracy and temporal consistency in 3D human pose estimation.
  arXiv  Detail & Related papers  (2025-08-04T04:50:05Z)
• Toward a Real-Time Framework for Accurate Monocular 3D Human Pose Estimation with Geometric Priors [0.0]
  We propose a framework that combines real-time 2D keypoint detection with geometry-aware 2D-to-3D lifting.<n>We discuss how these ingredients can enable fast, personalized, and accurate 3D pose estimation from monocular images without requiring specialized hardware.
  arXiv  Detail & Related papers  (2025-07-21T08:18:23Z)
• Bring Your Rear Cameras for Egocentric 3D Human Pose Estimation [69.68568832269285]
  Egocentric 3D human pose estimation has been actively studied using cameras installed in front of a head-mounted device (HMD)<n>It remains unclear if the same holds for full-body tracking due to self-occlusion and limited field-of-view coverage.<n>We propose a new transformer-based method that refines 2D joint heatmap estimation with multi-view information and heatmap uncertainty.
  arXiv  Detail & Related papers  (2025-03-14T17:59:54Z)
• Hybrid 3D Human Pose Estimation with Monocular Video and Sparse IMUs [15.017274891943162]
  Temporal 3D human pose estimation from monocular videos is a challenging task in human-centered computer vision. Inertial sensor has been introduced to provide complementary source of information. It remains challenging to integrate heterogeneous sensor data for producing physically rational 3D human poses.
  arXiv  Detail & Related papers  (2024-04-27T09:02:42Z)
• UPose3D: Uncertainty-Aware 3D Human Pose Estimation with Cross-View and Temporal Cues [55.69339788566899]
  UPose3D is a novel approach for multi-view 3D human pose estimation. It improves robustness and flexibility without requiring direct 3D annotations.
  arXiv  Detail & Related papers  (2024-04-23T00:18:00Z)
• EventEgo3D: 3D Human Motion Capture from Egocentric Event Streams [59.77837807004765]
  This paper introduces a new problem, i.e., 3D human motion capture from an egocentric monocular event camera with a fisheye lens. Event streams have high temporal resolution and provide reliable cues for 3D human motion capture under high-speed human motions and rapidly changing illumination. Our EE3D demonstrates robustness and superior 3D accuracy compared to existing solutions while supporting real-time 3D pose update rates of 140Hz.
  arXiv  Detail & Related papers  (2024-04-12T17:59:47Z)
• STRIDE: Single-video based Temporally Continuous Occlusion-Robust 3D Pose Estimation [27.854074900345314]
  We propose STRIDE (Single-video based TempoRally contInuous Occlusion-Robust 3D Estimation Pose), a novel Test-Time Training (TTT) approach to fit a human motion prior to each video.<n>Our framework demonstrates flexibility by being model-agnostic, allowing us to use any off-the-shelf 3D pose estimation method for improving robustness and temporal consistency.
  arXiv  Detail & Related papers  (2023-12-24T11:05:10Z)
• Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors [71.29186299435423]
  We introduce (HPS) Human POSEitioning System, a method to recover the full 3D pose of a human registered with a 3D scan of the surrounding environment. We show that our optimization-based integration exploits the benefits of the two, resulting in pose accuracy free of drift. HPS could be used for VR/AR applications where humans interact with the scene without requiring direct line of sight with an external camera.
  arXiv  Detail & Related papers  (2021-03-31T17:58:31Z)
• Kinematic-Structure-Preserved Representation for Unsupervised 3D Human Pose Estimation [58.72192168935338]
  Generalizability of human pose estimation models developed using supervision on large-scale in-studio datasets remains questionable. We propose a novel kinematic-structure-preserved unsupervised 3D pose estimation framework, which is not restrained by any paired or unpaired weak supervisions. Our proposed model employs three consecutive differentiable transformations named as forward-kinematics, camera-projection and spatial-map transformation.
  arXiv  Detail & Related papers  (2020-06-24T23:56:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.