Rotation-invariant Mixed Graphical Model Network for 2D Hand Pose Estimation

• URL: http://arxiv.org/abs/2002.02033v1
• Date: Wed, 5 Feb 2020 23:05:09 GMT
• Title: Rotation-invariant Mixed Graphical Model Network for 2D Hand Pose Estimation
• Authors: Deying Kong, Haoyu Ma, Yifei Chen, Xiaohui Xie
• Abstract summary: We propose a new architecture named Rotation-invariant Mixed Graphical Model Network (R-MGMN) By integrating a rotation net, the R-MGMN is invariant to rotations of the hand in the image. We evaluate the R-MGMN on two public hand pose datasets.
• Score: 21.19641797725211
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we propose a new architecture named Rotation-invariant Mixed Graphical Model Network (R-MGMN) to solve the problem of 2D hand pose estimation from a monocular RGB image. By integrating a rotation net, the R-MGMN is invariant to rotations of the hand in the image. It also has a pool of graphical models, from which a combination of graphical models could be selected, conditioning on the input image. Belief propagation is performed on each graphical model separately, generating a set of marginal distributions, which are taken as the confidence maps of hand keypoint positions. Final confidence maps are obtained by aggregating these confidence maps together. We evaluate the R-MGMN on two public hand pose datasets. Experiment results show our model outperforms the state-of-the-art algorithm which is widely used in 2D hand pose estimation by a noticeable margin.

Related papers

• HandDiff: 3D Hand Pose Estimation with Diffusion on Image-Point Cloud [60.47544798202017]
  Hand pose estimation is a critical task in various human-computer interaction applications. This paper proposes HandDiff, a diffusion-based hand pose estimation model that iteratively denoises accurate hand pose conditioned on hand-shaped image-point clouds. Experimental results demonstrate that the proposed HandDiff significantly outperforms the existing approaches on four challenging hand pose benchmark datasets.
  arXiv  Detail & Related papers  (2024-04-04T02:15:16Z)
• Learning Gaussian Representation for Eye Fixation Prediction [54.88001757991433]
  Existing eye fixation prediction methods perform the mapping from input images to the corresponding dense fixation maps generated from raw fixation points. We introduce Gaussian Representation for eye fixation modeling. We design our framework upon some lightweight backbones to achieve real-time fixation prediction.
  arXiv  Detail & Related papers  (2024-03-21T20:28:22Z)
• MeMaHand: Exploiting Mesh-Mano Interaction for Single Image Two-Hand Reconstruction [19.82874341207336]
  We propose to reconstruct meshes and estimate MANO parameters of two hands from a single RGB image simultaneously. MMIB consists of one graph residual block to aggregate local information and two transformer encoders to model long-range dependencies. Experiments on the InterHand2.6M benchmark demonstrate promising results over the state-of-the-art hand reconstruction methods.
  arXiv  Detail & Related papers  (2023-03-28T04:06:02Z)
• HandNeRF: Neural Radiance Fields for Animatable Interacting Hands [122.32855646927013]
  We propose a novel framework to reconstruct accurate appearance and geometry with neural radiance fields (NeRF) for interacting hands. We conduct extensive experiments to verify the merits of our proposed HandNeRF and report a series of state-of-the-art results.
  arXiv  Detail & Related papers  (2023-03-24T06:19:19Z)
• Real-time Pose and Shape Reconstruction of Two Interacting Hands With a Single Depth Camera [79.41374930171469]
  We present a novel method for real-time pose and shape reconstruction of two strongly interacting hands. Our approach combines an extensive list of favorable properties, namely it is marker-less. We show state-of-the-art results in scenes that exceed the complexity level demonstrated by previous work.
  arXiv  Detail & Related papers  (2021-06-15T11:39:49Z)
• Skeleton-aware multi-scale heatmap regression for 2D hand pose estimation [1.0152838128195467]
  We propose a new deep learning-based framework that consists of two main modules. The former presents a segmentation-based approach to detect the hand skeleton and localize the hand bounding box. The second module regresses the 2D joint locations through a multi-scale heatmap regression approach.
  arXiv  Detail & Related papers  (2021-05-23T10:23:51Z)
• Extreme Rotation Estimation using Dense Correlation Volumes [73.35119461422153]
  We present a technique for estimating the relative 3D rotation of an RGB image pair in an extreme setting. We observe that, even when images do not overlap, there may be rich hidden cues as to their geometric relationship. We propose a network design that can automatically learn such implicit cues by comparing all pairs of points between the two input images.
  arXiv  Detail & Related papers  (2021-04-28T02:00:04Z)
• Parallel mesh reconstruction streams for pose estimation of interacting hands [2.0305676256390934]
  We present a new multi-stream 3D mesh reconstruction network (MSMR-Net) for hand pose estimation from a single RGB image. Our model consists of an image encoder followed by a mesh-convolution decoder composed of connected graph convolution layers.
  arXiv  Detail & Related papers  (2021-04-25T10:14:15Z)
• Modelling the Distribution of 3D Brain MRI using a 2D Slice VAE [66.63629641650572]
  We propose a method to model 3D MR brain volumes distribution by combining a 2D slice VAE with a Gaussian model that captures the relationships between slices. We also introduce a novel evaluation method for generated volumes that quantifies how well their segmentations match those of true brain anatomy.
  arXiv  Detail & Related papers  (2020-07-09T13:23:15Z)
• Adaptive Graphical Model Network for 2D Handpose Estimation [19.592024471753025]
  We propose a new architecture to tackle the task of 2D hand pose estimation from a monocular RGB image. The Adaptive Graphical Model Network (AGMN) consists of two branches of deep convolutional neural networks for calculating unary and pairwise potential functions. Our approach outperforms the state-of-the-art method used in 2D hand keypoints estimation by a notable margin on two public datasets.
  arXiv  Detail & Related papers  (2019-09-18T04:19:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.