EventEgoHands++: Event-based Egocentric 3D Hand Mesh Reconstruction with Real Dataset

• URL: http://arxiv.org/abs/2609.17189v1
• Date: Tue, 15 Sep 2026 13:46:02 GMT
• Title: EventEgoHands++: Event-based Egocentric 3D Hand Mesh Reconstruction with Real Dataset
• Abstract summary: We propose a framework for event-based 3D hand mesh reconstruction from an egocentric viewpoint.<n>The proposed method incorporates a Hand Detector that estimates instance-level bounding boxes and masks for both the left and right hands.<n>We also introduce Adaptive Attention, which dynamically gates the attention based on these detection results.
• Score: 3.8731899020772165
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: 3D hand mesh reconstruction is a challenging yet essential task for downstream applications, including human-robot interaction and AR/VR. Although conventional cameras have been widely adopted for this task, methods that rely on them struggle in low-light environments and under severe motion blur. To address these limitations, event-based cameras have recently attracted attention for their high dynamic range and high temporal resolution. However, applying event cameras to egocentric hand reconstruction remains challenging because camera wearer's motion produces dense background events that obscure hand-specific signals. Although the first egocentric event-based approach mitigates this issue using hand segmentation, its binary hand mask does not distinguish between left and right hands. As a result, the model lacks instance-level hand information and predicts both hands even when only one or neither hand is present. This limitation leads to incorrect inter-hand relationships and degraded reconstruction accuracy. In this paper, we propose EventEgoHands++, a framework for event-based 3D hand mesh reconstruction from an egocentric viewpoint. The proposed method incorporates a Hand Detector that estimates instance-level bounding boxes and masks for both the left and right hands. Moreover, we introduce Adaptive Attention, which dynamically gates the attention based on these detection results to accurately learn the spatial relationship and mutual interactions between the hands. To train and evaluate our framework, we extend the synthetic N-HOT3D dataset and newly construct EEH-R, the largest real-world event-based egocentric hand dataset to date, comprising approximately 1M annotated frames captured in environments including low-light conditions. Extensive experiments on both synthetic and real datasets demonstrate that our method consistently outperforms the baselines.

Related papers

• HandsOnWorld: Unconstrained Egocentric Video Generation with Camera-Disentangled Hand Control [51.913695868638634]
  We present HandsOnWorld, a framework for hand-controlled egocentric video generation that forgoes multi-view and marker-based motion capture, learning instead from unconstrained monocular video.<n>We build EgoVid-Pro, a dataset of clean, protagonist-only hand trajectories spanning 103K clips and roughly 12M frames across diverse everyday scenes.<n>We also propose the Plcker Hand Map, a 3D-aware control signal that extends Plcker-ray representations from camera rays to the hand surface, disentangling camera and hand motion at the representation level.
  arXiv  Detail & Related papers  (2026-07-02T12:14:02Z)
• EgoEV-HandPose: Egocentric 3D Hand Pose Estimation and Gesture Recognition with Stereo Event Cameras [27.55657128726312]
  EgoEV-HandPose is an end-to-end framework for joint 3D bimanual pose estimation and gesture recognition from stereo event streams.<n>EgoEVHands is the first large-scale real-world stereo event-camera dataset for egocentric hand perception.
  arXiv  Detail & Related papers  (2026-05-12T15:51:04Z)
• EventEgoHands: Event-based Egocentric 3D Hand Mesh Reconstruction [2.3695551082138864]
  Reconstructing 3D hand mesh is challenging but an important task for human-computer interaction and AR/VR applications.<n>We propose EventEgoHands, a novel method for event-based 3D hand mesh reconstruction in an egocentric view.<n>Our approach introduces a Hand Module that extracts hand regions, effectively mitigating the influence of dynamic background events.
  arXiv  Detail & Related papers  (2025-05-25T14:36:51Z)
• Benchmarks and Challenges in Pose Estimation for Egocentric Hand Interactions with Objects [89.95728475983263]
  holistic 3Dunderstanding of such interactions from egocentric views is important for tasks in robotics, AR/VR, action recognition and motion generation. We design the HANDS23 challenge based on the AssemblyHands and ARCTIC datasets with carefully designed training and testing splits. Based on the results of the top submitted methods and more recent baselines on the leaderboards, we perform a thorough analysis on 3D hand(-object) reconstruction tasks.
  arXiv  Detail & Related papers  (2024-03-25T05:12:21Z)
• 3D Pose Estimation of Two Interacting Hands from a Monocular Event Camera [59.846927201816776]
  This paper introduces the first framework for 3D tracking of two fast-moving and interacting hands from a single monocular event camera. Our approach tackles the left-right hand ambiguity with a novel semi-supervised feature-wise attention mechanism and integrates an intersection loss to fix hand collisions.
  arXiv  Detail & Related papers  (2023-12-21T18:59:57Z)
• SHOWMe: Benchmarking Object-agnostic Hand-Object 3D Reconstruction [13.417086460511696]
  We introduce the SHOWMe dataset which consists of 96 videos, annotated with real and detailed hand-object 3D textured meshes. We consider a rigid hand-object scenario, in which the pose of the hand with respect to the object remains constant during the whole video sequence. This assumption allows us to register sub-millimetre-precise groundtruth 3D scans to the image sequences in SHOWMe.
  arXiv  Detail & Related papers  (2023-09-19T16:48:29Z)
• EvHandPose: Event-based 3D Hand Pose Estimation with Sparse Supervision [50.060055525889915]
  Event camera shows great potential in 3D hand pose estimation, especially addressing the challenges of fast motion and high dynamic range in a low-power way. It is challenging to design event representation to encode hand motion information especially when the hands are not moving. In this paper, we propose EvHandPose with novel hand flow representations in Event-to-Pose module for accurate hand pose estimation.
  arXiv  Detail & Related papers  (2023-03-06T03:27:17Z)
• 3D Interacting Hand Pose Estimation by Hand De-occlusion and Removal [85.30756038989057]
  Estimating 3D interacting hand pose from a single RGB image is essential for understanding human actions. We propose to decompose the challenging interacting hand pose estimation task and estimate the pose of each hand separately. Experiments show that the proposed method significantly outperforms previous state-of-the-art interacting hand pose estimation approaches.
  arXiv  Detail & Related papers  (2022-07-22T13:04:06Z)
• Monocular 3D Reconstruction of Interacting Hands via Collision-Aware Factorized Refinements [96.40125818594952]
  We make the first attempt to reconstruct 3D interacting hands from monocular single RGB images. Our method can generate 3D hand meshes with both precise 3D poses and minimal collisions.
  arXiv  Detail & Related papers  (2021-11-01T08:24:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.