Nonparametric Structure Regularization Machine for 2D Hand Pose Estimation

• URL: http://arxiv.org/abs/2001.08869v1
• Date: Fri, 24 Jan 2020 03:27:32 GMT
• Title: Nonparametric Structure Regularization Machine for 2D Hand Pose Estimation
• Authors: Yifei Chen, Haoyu Ma, Deying Kong, Xiangyi Yan, Jianbao Wu, Wei Fan, Xiaohui Xie
• Abstract summary: Hand pose estimation is more challenging than body pose estimation due to severe articulation, self-occlusion and high dexterity of the hand. We propose a novel Nonparametric Structure Regularization Machine (NSRM) for 2D hand pose estimation, adopting a cascade multi-task architecture to learn hand structure and keypoint representations jointly.
• Score: 21.250031729596085
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Hand pose estimation is more challenging than body pose estimation due to severe articulation, self-occlusion and high dexterity of the hand. Current approaches often rely on a popular body pose algorithm, such as the Convolutional Pose Machine (CPM), to learn 2D keypoint features. These algorithms cannot adequately address the unique challenges of hand pose estimation, because they are trained solely based on keypoint positions without seeking to explicitly model structural relationship between them. We propose a novel Nonparametric Structure Regularization Machine (NSRM) for 2D hand pose estimation, adopting a cascade multi-task architecture to learn hand structure and keypoint representations jointly. The structure learning is guided by synthetic hand mask representations, which are directly computed from keypoint positions, and is further strengthened by a novel probabilistic representation of hand limbs and an anatomically inspired composition strategy of mask synthesis. We conduct extensive studies on two public datasets - OneHand 10k and CMU Panoptic Hand. Experimental results demonstrate that explicitly enforcing structure learning consistently improves pose estimation accuracy of CPM baseline models, by 1.17% on the first dataset and 4.01% on the second one. The implementation and experiment code is freely available online. Our proposal of incorporating structural learning to hand pose estimation requires no additional training information, and can be a generic add-on module to other pose estimation models.

Related papers

• HandDiff: 3D Hand Pose Estimation with Diffusion on Image-Point Cloud [60.47544798202017]
  Hand pose estimation is a critical task in various human-computer interaction applications. This paper proposes HandDiff, a diffusion-based hand pose estimation model that iteratively denoises accurate hand pose conditioned on hand-shaped image-point clouds. Experimental results demonstrate that the proposed HandDiff significantly outperforms the existing approaches on four challenging hand pose benchmark datasets.
  arXiv  Detail & Related papers  (2024-04-04T02:15:16Z)
• Self-supervised Optimization of Hand Pose Estimation using Anatomical Features and Iterative Learning [4.698846136465861]
  This paper presents a self-supervised pipeline for adapting hand pose estimation to specific use cases with minimal human interaction. The pipeline consists of a general machine learning model for hand pose estimation trained on a generalized dataset. The effectiveness of the pipeline is demonstrated by training an activity recognition as a downstream task in the manual assembly scenario.
  arXiv  Detail & Related papers  (2023-07-06T14:13:11Z)
• Human Pose as Compositional Tokens [88.28348144244131]
  We present a structured representation, named Pose as Compositional Tokens (PCT), to explore the joint dependency. It represents a pose by M discrete tokens with each characterizing a sub-structure with several interdependent joints. A pre-learned decoder network is used to recover the pose from the tokens without further post-processing.
  arXiv  Detail & Related papers  (2023-03-21T07:14:18Z)
• Im2Hands: Learning Attentive Implicit Representation of Interacting Two-Hand Shapes [58.551154822792284]
  Implicit Two Hands (Im2Hands) is the first neural implicit representation of two interacting hands. Im2Hands can produce fine-grained geometry of two hands with high hand-to-hand and hand-to-image coherency. We experimentally demonstrate the effectiveness of Im2Hands on two-hand reconstruction in comparison to related methods.
  arXiv  Detail & Related papers  (2023-02-28T06:38:25Z)
• 2D Human Pose Estimation with Explicit Anatomical Keypoints Structure Constraints [15.124606575017621]
  We present a novel 2D human pose estimation method with explicit anatomical keypoints structure constraints. Our proposed model can be plugged in the most existing bottom-up or top-down human pose estimation methods. Our methods perform favorably against the most existing bottom-up and top-down human pose estimation methods.
  arXiv  Detail & Related papers  (2022-12-05T11:01:43Z)
• 3D Interacting Hand Pose Estimation by Hand De-occlusion and Removal [85.30756038989057]
  Estimating 3D interacting hand pose from a single RGB image is essential for understanding human actions. We propose to decompose the challenging interacting hand pose estimation task and estimate the pose of each hand separately. Experiments show that the proposed method significantly outperforms previous state-of-the-art interacting hand pose estimation approaches.
  arXiv  Detail & Related papers  (2022-07-22T13:04:06Z)
• Real-time Pose and Shape Reconstruction of Two Interacting Hands With a Single Depth Camera [79.41374930171469]
  We present a novel method for real-time pose and shape reconstruction of two strongly interacting hands. Our approach combines an extensive list of favorable properties, namely it is marker-less. We show state-of-the-art results in scenes that exceed the complexity level demonstrated by previous work.
  arXiv  Detail & Related papers  (2021-06-15T11:39:49Z)
• Skeleton-aware multi-scale heatmap regression for 2D hand pose estimation [1.0152838128195467]
  We propose a new deep learning-based framework that consists of two main modules. The former presents a segmentation-based approach to detect the hand skeleton and localize the hand bounding box. The second module regresses the 2D joint locations through a multi-scale heatmap regression approach.
  arXiv  Detail & Related papers  (2021-05-23T10:23:51Z)
• Self-Supervised 3D Human Pose Estimation via Part Guided Novel Image Synthesis [72.34794624243281]
  We propose a self-supervised learning framework to disentangle variations from unlabeled video frames. Our differentiable formalization, bridging the representation gap between the 3D pose and spatial part maps, allows us to operate on videos with diverse camera movements.
  arXiv  Detail & Related papers  (2020-04-09T07:55:01Z)
• Robust, Occlusion-aware Pose Estimation for Objects Grasped by Adaptive Hands [16.343365158924183]
  manipulation tasks, such as within-hand manipulation, require the object's pose relative to a robot hand. This paper presents a depth-based framework, which aims for robust pose estimation and short response times.
  arXiv  Detail & Related papers  (2020-03-07T05:51:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.