Skeleton-aware multi-scale heatmap regression for 2D hand pose estimation

• URL: http://arxiv.org/abs/2105.10904v1
• Date: Sun, 23 May 2021 10:23:51 GMT
• Title: Skeleton-aware multi-scale heatmap regression for 2D hand pose estimation
• Authors: Ikram Kourbane, Yakup Genc
• Abstract summary: We propose a new deep learning-based framework that consists of two main modules. The former presents a segmentation-based approach to detect the hand skeleton and localize the hand bounding box. The second module regresses the 2D joint locations through a multi-scale heatmap regression approach.
• Score: 1.0152838128195467
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Existing RGB-based 2D hand pose estimation methods learn the joint locations from a single resolution, which is not suitable for different hand sizes. To tackle this problem, we propose a new deep learning-based framework that consists of two main modules. The former presents a segmentation-based approach to detect the hand skeleton and localize the hand bounding box. The second module regresses the 2D joint locations through a multi-scale heatmap regression approach that exploits the predicted hand skeleton as a constraint to guide the model. Furthermore, we construct a new dataset that is suitable for both hand detection and pose estimation. We qualitatively and quantitatively validate our method on two datasets. Results demonstrate that the proposed method outperforms state-of-the-art and can recover the pose even in cluttered images and complex poses.

Related papers

• HandDiff: 3D Hand Pose Estimation with Diffusion on Image-Point Cloud [60.47544798202017]
  Hand pose estimation is a critical task in various human-computer interaction applications. This paper proposes HandDiff, a diffusion-based hand pose estimation model that iteratively denoises accurate hand pose conditioned on hand-shaped image-point clouds. Experimental results demonstrate that the proposed HandDiff significantly outperforms the existing approaches on four challenging hand pose benchmark datasets.
  arXiv  Detail & Related papers  (2024-04-04T02:15:16Z)
• HandNeRF: Neural Radiance Fields for Animatable Interacting Hands [122.32855646927013]
  We propose a novel framework to reconstruct accurate appearance and geometry with neural radiance fields (NeRF) for interacting hands. We conduct extensive experiments to verify the merits of our proposed HandNeRF and report a series of state-of-the-art results.
  arXiv  Detail & Related papers  (2023-03-24T06:19:19Z)
• Im2Hands: Learning Attentive Implicit Representation of Interacting Two-Hand Shapes [58.551154822792284]
  Implicit Two Hands (Im2Hands) is the first neural implicit representation of two interacting hands. Im2Hands can produce fine-grained geometry of two hands with high hand-to-hand and hand-to-image coherency. We experimentally demonstrate the effectiveness of Im2Hands on two-hand reconstruction in comparison to related methods.
  arXiv  Detail & Related papers  (2023-02-28T06:38:25Z)
• 3D Interacting Hand Pose Estimation by Hand De-occlusion and Removal [85.30756038989057]
  Estimating 3D interacting hand pose from a single RGB image is essential for understanding human actions. We propose to decompose the challenging interacting hand pose estimation task and estimate the pose of each hand separately. Experiments show that the proposed method significantly outperforms previous state-of-the-art interacting hand pose estimation approaches.
  arXiv  Detail & Related papers  (2022-07-22T13:04:06Z)
• Poseur: Direct Human Pose Regression with Transformers [119.79232258661995]
  We propose a direct, regression-based approach to 2D human pose estimation from single images. Our framework is end-to-end differentiable, and naturally learns to exploit the dependencies between keypoints. Ours is the first regression-based approach to perform favorably compared to the best heatmap-based pose estimation methods.
  arXiv  Detail & Related papers  (2022-01-19T04:31:57Z)
• Real-time Pose and Shape Reconstruction of Two Interacting Hands With a Single Depth Camera [79.41374930171469]
  We present a novel method for real-time pose and shape reconstruction of two strongly interacting hands. Our approach combines an extensive list of favorable properties, namely it is marker-less. We show state-of-the-art results in scenes that exceed the complexity level demonstrated by previous work.
  arXiv  Detail & Related papers  (2021-06-15T11:39:49Z)
• Rotation-invariant Mixed Graphical Model Network for 2D Hand Pose Estimation [21.19641797725211]
  We propose a new architecture named Rotation-invariant Mixed Graphical Model Network (R-MGMN) By integrating a rotation net, the R-MGMN is invariant to rotations of the hand in the image. We evaluate the R-MGMN on two public hand pose datasets.
  arXiv  Detail & Related papers  (2020-02-05T23:05:09Z)
• Adaptive Graphical Model Network for 2D Handpose Estimation [19.592024471753025]
  We propose a new architecture to tackle the task of 2D hand pose estimation from a monocular RGB image. The Adaptive Graphical Model Network (AGMN) consists of two branches of deep convolutional neural networks for calculating unary and pairwise potential functions. Our approach outperforms the state-of-the-art method used in 2D hand keypoints estimation by a notable margin on two public datasets.
  arXiv  Detail & Related papers  (2019-09-18T04:19:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.