EgoForce: Forearm-Guided Camera-Space 3D Hand Pose from a Monocular Egocentric Camera
- URL: http://arxiv.org/abs/2605.12498v1
- Date: Tue, 12 May 2026 17:59:56 GMT
- Title: EgoForce: Forearm-Guided Camera-Space 3D Hand Pose from a Monocular Egocentric Camera
- Abstract summary: EgoForce is a monocular 3D hand reconstruction framework that recovers robust, absolute 3D hand pose.<n>It operates across fisheye, perspective, and distorted wide-FOV camera models using a single unified network.<n>EgoForce achieves state-of-the-art 3D accuracy, reducing camera-space MPJPE by up to 28%.
- Score: 40.28396219356337
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Reconstructing the absolute 3D pose and shape of the hands from the user's viewpoint using a single head-mounted camera is crucial for practical egocentric interaction in AR/VR, telepresence, and hand-centric manipulation tasks, where sensing must remain compact and unobtrusive. While monocular RGB methods have made progress, they remain constrained by depth-scale ambiguity and struggle to generalize across the diverse optical configurations of head-mounted devices. As a result, models typically require extensive training on device-specific datasets, which are costly and laborious to acquire. This paper addresses these challenges by introducing EgoForce, a monocular 3D hand reconstruction framework that recovers robust, absolute 3D hand pose and its position from the user's (camera-space) viewpoint. EgoForce operates across fisheye, perspective, and distorted wide-FOV camera models using a single unified network. Our approach combines a differentiable forearm representation that stabilizes hand pose, a unified arm-hand transformer that predicts both hand and forearm geometry from a single egocentric view, mitigating depth-scale ambiguity, and a ray space closed-form solver that enables absolute 3D pose recovery across diverse head-mounted camera models. Experiments on three egocentric benchmarks show that EgoForce achieves state-of-the-art 3D accuracy, reducing camera-space MPJPE by up to 28% on the HOT3D dataset compared to prior methods and maintaining consistent performance across camera configurations. For more details, visit the project page at https://dfki-av.github.io/EgoForce.
Related papers
- HandsOnWorld: Unconstrained Egocentric Video Generation with Camera-Disentangled Hand Control [51.913695868638634]
We present HandsOnWorld, a framework for hand-controlled egocentric video generation that forgoes multi-view and marker-based motion capture, learning instead from unconstrained monocular video.<n>We build EgoVid-Pro, a dataset of clean, protagonist-only hand trajectories spanning 103K clips and roughly 12M frames across diverse everyday scenes.<n>We also propose the Plcker Hand Map, a 3D-aware control signal that extends Plcker-ray representations from camera rays to the hand surface, disentangling camera and hand motion at the representation level.
arXiv Detail & Related papers (2026-07-02T12:14:02Z) - EgoEV-HandPose: Egocentric 3D Hand Pose Estimation and Gesture Recognition with Stereo Event Cameras [27.55657128726312]
EgoEV-HandPose is an end-to-end framework for joint 3D bimanual pose estimation and gesture recognition from stereo event streams.<n>EgoEVHands is the first large-scale real-world stereo event-camera dataset for egocentric hand perception.
arXiv Detail & Related papers (2026-05-12T15:51:04Z) - Controllable Egocentric Video Generation via Occlusion-Aware Sparse 3D Hand Joints [87.13154261503168]
Motion-controllable video generation is crucial for egocentric applications in virtual reality and embodied AI.<n>Existing methods often struggle to achieve 3D-consistent fine-grained hand articulation.<n>We propose a novel framework that generates egocentric videos from a single reference frame.
arXiv Detail & Related papers (2026-03-12T10:02:23Z) - Ego-Exo 3D Hand Tracking in the Wild with a Mobile Multi-Camera Rig [14.496137517475743]
We introduce a novel marker-less multi-camera system designed to capture precise 3D hands and objects.<n>We combine a lightweight, back-mounted capture rig with eight exocentric cameras, and a user-worn Meta Quest 3 headset, which contributes two egocentric views.<n>We collect an annotated dataset featuring synchronized multi-view images and precise 3D hand poses.
arXiv Detail & Related papers (2025-10-02T22:26:03Z) - Bring Your Rear Cameras for Egocentric 3D Human Pose Estimation [69.68568832269285]
Egocentric 3D human pose estimation has been actively studied using cameras installed in front of a head-mounted device (HMD)<n>It remains unclear if the same holds for full-body tracking due to self-occlusion and limited field-of-view coverage.<n>We propose a new transformer-based method that refines 2D joint heatmap estimation with multi-view information and heatmap uncertainty.
arXiv Detail & Related papers (2025-03-14T17:59:54Z) - Decaf: Monocular Deformation Capture for Face and Hand Interactions [77.75726740605748]
This paper introduces the first method that allows tracking human hands interacting with human faces in 3D from single monocular RGB videos.
We model hands as articulated objects inducing non-rigid face deformations during an active interaction.
Our method relies on a new hand-face motion and interaction capture dataset with realistic face deformations acquired with a markerless multi-view camera system.
arXiv Detail & Related papers (2023-09-28T17:59:51Z) - Consistent 3D Hand Reconstruction in Video via self-supervised Learning [67.55449194046996]
We present a method for reconstructing accurate and consistent 3D hands from a monocular video.
detected 2D hand keypoints and the image texture provide important cues about the geometry and texture of the 3D hand.
We propose $rm S2HAND$, a self-supervised 3D hand reconstruction model.
arXiv Detail & Related papers (2022-01-24T09:44:11Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.