UniHands: Unifying Various Wild-Collected Keypoints for Personalized Hand Reconstruction

• URL: http://arxiv.org/abs/2411.11845v1
• Date: Mon, 18 Nov 2024 18:59:58 GMT
• Title: UniHands: Unifying Various Wild-Collected Keypoints for Personalized Hand Reconstruction
• Authors: Menghe Zhang, Joonyeoup Kim, Yangwen Liang, Shuangquan Wang, Kee-Bong Song,
• Abstract summary: We present UniHands, a novel method for creating standardized yet personalized hand models. Unlike existing neural implicit representation methods, UniHands uses the widely-adopted parametric models MANO and NIMBLE. It also derives unified hand joints from the meshes, which facilitates seamless integration into various hand-related tasks.
• Score: 4.0025708029346445
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accurate hand motion capture and standardized 3D representation are essential for various hand-related tasks. Collecting keypoints-only data, while efficient and cost-effective, results in low-fidelity representations and lacks surface information. Furthermore, data inconsistencies across sources challenge their integration and use. We present UniHands, a novel method for creating standardized yet personalized hand models from wild-collected keypoints from diverse sources. Unlike existing neural implicit representation methods, UniHands uses the widely-adopted parametric models MANO and NIMBLE, providing a more scalable and versatile solution. It also derives unified hand joints from the meshes, which facilitates seamless integration into various hand-related tasks. Experiments on the FreiHAND and InterHand2.6M datasets demonstrate its ability to precisely reconstruct hand mesh vertices and keypoints, effectively capturing high-degree articulation motions. Empirical studies involving nine participants show a clear preference for our unified joints over existing configurations for accuracy and naturalism (p-value 0.016).

Related papers

• Body-Hand Modality Expertized Networks with Cross-attention for Fine-grained Skeleton Action Recognition [28.174638880324014]
  BHaRNet is a novel framework that augments a typical body-expert model with a hand-expert model. Our model jointly trains both streams with an ensemble loss that fosters cooperative specialization. Inspired by MMNet, we also demonstrate the applicability of our approach to multi-modal tasks by leveraging RGB information.
  arXiv  Detail & Related papers  (2025-03-19T07:54:52Z)
• UniHOPE: A Unified Approach for Hand-Only and Hand-Object Pose Estimation [82.93208597526503]
  Existing methods are specialized, focusing on either bare-hand or hand interacting with object. No method can flexibly handle both scenarios and their performance degrades when applied to the other scenario. We propose UniHOPE, a unified approach for general 3D hand-object pose estimation.
  arXiv  Detail & Related papers  (2025-03-17T15:46:43Z)
• HanDrawer: Leveraging Spatial Information to Render Realistic Hands Using a Conditional Diffusion Model in Single Stage [16.890823951821396]
  We propose HanDrawer, a module to condition the hand generation process. The spatially fused features are used to guide a single stage diffusion model denoising process. HanDrawer learns the entire image features while paying special attention to the hand region.
  arXiv  Detail & Related papers  (2025-03-03T23:29:33Z)
• Learning Interaction-aware 3D Gaussian Splatting for One-shot Hand Avatars [47.61442517627826]
  We propose to create animatable avatars for interacting hands with 3D Gaussian Splatting (GS) and single-image inputs. Our proposed method is validated via extensive experiments on the large-scale InterHand2.6M dataset.
  arXiv  Detail & Related papers  (2024-10-11T14:14:51Z)
• Decomposed Vector-Quantized Variational Autoencoder for Human Grasp Generation [27.206656215734295]
  We propose a novel Decomposed Vector-Quantized Variational Autoencoder (DVQ-VAE) to generate realistic human grasps. Part-aware decomposed architecture facilitates more precise management of the interaction between each component of hand and object. Our model achieved about 14.1% relative improvement in the quality index compared to the state-of-the-art methods in four widely-adopted benchmarks.
  arXiv  Detail & Related papers  (2024-07-19T06:41:16Z)
• HandDiff: 3D Hand Pose Estimation with Diffusion on Image-Point Cloud [60.47544798202017]
  Hand pose estimation is a critical task in various human-computer interaction applications. This paper proposes HandDiff, a diffusion-based hand pose estimation model that iteratively denoises accurate hand pose conditioned on hand-shaped image-point clouds. Experimental results demonstrate that the proposed HandDiff significantly outperforms the existing approaches on four challenging hand pose benchmark datasets.
  arXiv  Detail & Related papers  (2024-04-04T02:15:16Z)
• HandBooster: Boosting 3D Hand-Mesh Reconstruction by Conditional Synthesis and Sampling of Hand-Object Interactions [68.28684509445529]
  We present HandBooster, a new approach to uplift the data diversity and boost the 3D hand-mesh reconstruction performance. First, we construct versatile content-aware conditions to guide a diffusion model to produce realistic images with diverse hand appearances, poses, views, and backgrounds. Then, we design a novel condition creator based on our similarity-aware distribution sampling strategies to deliberately find novel and realistic interaction poses that are distinctive from the training set.
  arXiv  Detail & Related papers  (2024-03-27T13:56:08Z)
• 3D Hand Reconstruction via Aggregating Intra and Inter Graphs Guided by Prior Knowledge for Hand-Object Interaction Scenario [8.364378460776832]
  We propose a 3D hand reconstruction network combining the benefits of model-based and model-free approaches to balance accuracy and physical plausibility for hand-object interaction scenario. Firstly, we present a novel MANO pose parameters regression module from 2D joints directly, which avoids the process of highly nonlinear mapping from abstract image feature.
  arXiv  Detail & Related papers  (2024-03-04T05:11:26Z)
• Interacting Hand-Object Pose Estimation via Dense Mutual Attention [97.26400229871888]
  3D hand-object pose estimation is the key to the success of many computer vision applications. We propose a novel dense mutual attention mechanism that is able to model fine-grained dependencies between the hand and the object. Our method is able to produce physically plausible poses with high quality and real-time inference speed.
  arXiv  Detail & Related papers  (2022-11-16T10:01:33Z)
• A Skeleton-Driven Neural Occupancy Representation for Articulated Hands [49.956892429789775]
  Hand ArticuLated Occupancy (HALO) is a novel representation of articulated hands that bridges the advantages of 3D keypoints and neural implicit surfaces. We demonstrate the applicability of HALO to the task of conditional generation of hands that grasp 3D objects.
  arXiv  Detail & Related papers  (2021-09-23T14:35:19Z)
• HandFoldingNet: A 3D Hand Pose Estimation Network Using Multiscale-Feature Guided Folding of a 2D Hand Skeleton [4.1954750695245835]
  This paper proposes HandFoldingNet, an accurate and efficient hand pose estimator. The proposed model utilizes a folding-based decoder that folds a given 2D hand skeleton into the corresponding joint coordinates. Experimental results show that the proposed model outperforms the existing methods on three hand pose benchmark datasets.
  arXiv  Detail & Related papers  (2021-08-12T05:52:44Z)
• Measuring Generalisation to Unseen Viewpoints, Articulations, Shapes and Objects for 3D Hand Pose Estimation under Hand-Object Interaction [137.28465645405655]
  HANDS'19 is a challenge to evaluate the abilities of current 3D hand pose estimators (HPEs) to interpolate and extrapolate the poses of a training set. We show that the accuracy of state-of-the-art methods can drop, and that they fail mostly on poses absent from the training set.
  arXiv  Detail & Related papers  (2020-03-30T19:28:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.