OccRobNet : Occlusion Robust Network for Accurate 3D Interacting Hand-Object Pose Estimation

• URL: http://arxiv.org/abs/2503.21723v1
• Date: Thu, 27 Mar 2025 17:36:55 GMT
• Title: OccRobNet : Occlusion Robust Network for Accurate 3D Interacting Hand-Object Pose Estimation
• Authors: Mallika Garg, Debashis Ghosh, Pyari Mohan Pradhan,
• Abstract summary: Occlusion is one of the challenging issues when estimating 3D hand pose.<n>In this paper, we propose an occlusion robust and accurate method for the estimation of 3D hand-object pose from the input RGB image.<n>Our method includes first localising the hand joints using a CNN based model and then refining them by extracting contextual information.
• Score: 5.311735227179715
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Occlusion is one of the challenging issues when estimating 3D hand pose. This problem becomes more prominent when hand interacts with an object or two hands are involved. In the past works, much attention has not been given to these occluded regions. But these regions contain important and beneficial information that is vital for 3D hand pose estimation. Thus, in this paper, we propose an occlusion robust and accurate method for the estimation of 3D hand-object pose from the input RGB image. Our method includes first localising the hand joints using a CNN based model and then refining them by extracting contextual information. The self attention transformer then identifies the specific joints along with the hand identity. This helps the model to identify the hand belongingness of a particular joint which helps to detect the joint even in the occluded region. Further, these joints with hand identity are then used to estimate the pose using cross attention mechanism. Thus, by identifying the joints in the occluded region, the obtained network becomes robust to occlusion. Hence, this network achieves state-of-the-art results when evaluated on the InterHand2.6M, HO3D and H$_2$O3D datasets.

Related papers

• SHARP: Segmentation of Hands and Arms by Range using Pseudo-Depth for Enhanced Egocentric 3D Hand Pose Estimation and Action Recognition [5.359837526794863]
  Hand pose represents key information for action recognition in the egocentric perspective. We propose to improve egocentric 3D hand pose estimation based on RGB frames only by using pseudo-depth images.
  arXiv  Detail & Related papers  (2024-08-19T14:30:29Z)
• HandDiff: 3D Hand Pose Estimation with Diffusion on Image-Point Cloud [60.47544798202017]
  Hand pose estimation is a critical task in various human-computer interaction applications. This paper proposes HandDiff, a diffusion-based hand pose estimation model that iteratively denoises accurate hand pose conditioned on hand-shaped image-point clouds. Experimental results demonstrate that the proposed HandDiff significantly outperforms the existing approaches on four challenging hand pose benchmark datasets.
  arXiv  Detail & Related papers  (2024-04-04T02:15:16Z)
• HOISDF: Constraining 3D Hand-Object Pose Estimation with Global Signed Distance Fields [96.04424738803667]
  HOISDF is a guided hand-object pose estimation network. It exploits hand and object SDFs to provide a global, implicit representation over the complete reconstruction volume. We show that HOISDF achieves state-of-the-art results on hand-object pose estimation benchmarks.
  arXiv  Detail & Related papers  (2024-02-26T22:48:37Z)
• 3D Interacting Hand Pose Estimation by Hand De-occlusion and Removal [85.30756038989057]
  Estimating 3D interacting hand pose from a single RGB image is essential for understanding human actions. We propose to decompose the challenging interacting hand pose estimation task and estimate the pose of each hand separately. Experiments show that the proposed method significantly outperforms previous state-of-the-art interacting hand pose estimation approaches.
  arXiv  Detail & Related papers  (2022-07-22T13:04:06Z)
• HandOccNet: Occlusion-Robust 3D Hand Mesh Estimation Network [57.206129938611454]
  We propose a novel 3D hand mesh estimation network HandOccNet. By injecting the hand information to the occluded region, our HandOccNet reaches the state-of-the-art performance on 3D hand mesh benchmarks.
  arXiv  Detail & Related papers  (2022-03-28T08:12:16Z)
• HandsFormer: Keypoint Transformer for Monocular 3D Pose Estimation ofHands and Object in Interaction [33.661745138578596]
  We propose a robust and accurate method for estimating the 3D poses of two hands in close interaction from a single color image. Our method starts by extracting a set of potential 2D locations for the joints of both hands as extrema of a heatmap. We use appearance and spatial encodings of these locations as input to a transformer, and leverage the attention mechanisms to sort out the correct configuration of the joints.
  arXiv  Detail & Related papers  (2021-04-29T20:19:20Z)
• InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Image [71.17227941339935]
  We propose a large-scale dataset, InterHand2.6M, and a network, InterNet, for 3D interacting hand pose estimation from a single RGB image. In our experiments, we demonstrate big gains in 3D interacting hand pose estimation accuracy when leveraging the interacting hand data in InterHand2.6M. We also report the accuracy of InterNet on InterHand2.6M, which serves as a strong baseline for this new dataset.
  arXiv  Detail & Related papers  (2020-08-21T05:15:58Z)
• Two-hand Global 3D Pose Estimation Using Monocular RGB [0.0]
  We tackle the challenging task of estimating global 3D joint locations for both hands via only monocular RGB input images. We propose a novel multi-stage convolutional neural network based pipeline that accurately segments and locates the hands. We present the first work that achieves accurate global 3D hand tracking on both hands using RGB-only inputs.
  arXiv  Detail & Related papers  (2020-06-01T23:53:52Z)
• Measuring Generalisation to Unseen Viewpoints, Articulations, Shapes and Objects for 3D Hand Pose Estimation under Hand-Object Interaction [137.28465645405655]
  HANDS'19 is a challenge to evaluate the abilities of current 3D hand pose estimators (HPEs) to interpolate and extrapolate the poses of a training set. We show that the accuracy of state-of-the-art methods can drop, and that they fail mostly on poses absent from the training set.
  arXiv  Detail & Related papers  (2020-03-30T19:28:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.