Recovering 3D Hand Mesh Sequence from a Single Blurry Image: A New Dataset and Temporal Unfolding

• URL: http://arxiv.org/abs/2303.15417v1
• Date: Mon, 27 Mar 2023 17:40:29 GMT
• Title: Recovering 3D Hand Mesh Sequence from a Single Blurry Image: A New Dataset and Temporal Unfolding
• Authors: Yeonguk Oh, JoonKyu Park, Jaeha Kim, Gyeongsik Moon, Kyoung Mu Lee
• Abstract summary: We first present a novel dataset BlurHand, which contains blurry hand images with 3D groundtruths. The BlurHand is constructed by synthesizing motion blur from sequential sharp hand images, imitating realistic and natural motion blurs. In addition to the new dataset, we propose BlurHandNet, a baseline network for accurate 3D hand mesh recovery from a blurry hand image.
• Score: 54.49373038369293
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Hands, one of the most dynamic parts of our body, suffer from blur due to their active movements. However, previous 3D hand mesh recovery methods have mainly focused on sharp hand images rather than considering blur due to the absence of datasets providing blurry hand images. We first present a novel dataset BlurHand, which contains blurry hand images with 3D groundtruths. The BlurHand is constructed by synthesizing motion blur from sequential sharp hand images, imitating realistic and natural motion blurs. In addition to the new dataset, we propose BlurHandNet, a baseline network for accurate 3D hand mesh recovery from a blurry hand image. Our BlurHandNet unfolds a blurry input image to a 3D hand mesh sequence to utilize temporal information in the blurry input image, while previous works output a static single hand mesh. We demonstrate the usefulness of BlurHand for the 3D hand mesh recovery from blurry images in our experiments. The proposed BlurHandNet produces much more robust results on blurry images while generalizing well to in-the-wild images. The training codes and BlurHand dataset are available at https://github.com/JaehaKim97/BlurHand_RELEASE.

Related papers

• AttentionHand: Text-driven Controllable Hand Image Generation for 3D Hand Reconstruction in the Wild [18.351368674337134]
  AttentionHand is a novel method for text-driven controllable hand image generation. It can generate various and numerous in-the-wild hand images well-aligned with 3D hand label. It achieves state-of-the-art among text-to-hand image generation models.
  arXiv  Detail & Related papers  (2024-07-25T13:29:32Z)
• HandGCAT: Occlusion-Robust 3D Hand Mesh Reconstruction from Monocular Images [9.554136347258057]
  We propose a robust and accurate method for reconstructing 3D hand mesh from monocular images. HandGCAT can fully exploit hand prior as compensation information to enhance occluded region features.
  arXiv  Detail & Related papers  (2024-02-27T03:40:43Z)
• HandNeRF: Learning to Reconstruct Hand-Object Interaction Scene from a Single RGB Image [41.580285338167315]
  This paper presents a method to learn hand-object interaction prior for reconstructing a 3D hand-object scene from a single RGB image. We use the hand shape to constrain the possible relative configuration of the hand and object geometry. We show that HandNeRF is able to reconstruct hand-object scenes of novel grasp configurations more accurately than comparable methods.
  arXiv  Detail & Related papers  (2023-09-14T17:42:08Z)
• Human from Blur: Human Pose Tracking from Blurry Images [89.65036443997103]
  We propose a method to estimate 3D human poses from substantially blurred images. Key idea is to tackle the inverse problem of image deblurring by modeling the forward problem with a 3D human model, a texture map, and a sequence of poses to describe human motion. Using a differentiable step, we can solve the inverse problem by backpropagating the pixel-wise reprojection error to recover the best human motion representation.
  arXiv  Detail & Related papers  (2023-03-30T08:05:59Z)
• Monocular 3D Reconstruction of Interacting Hands via Collision-Aware Factorized Refinements [96.40125818594952]
  We make the first attempt to reconstruct 3D interacting hands from monocular single RGB images. Our method can generate 3D hand meshes with both precise 3D poses and minimal collisions.
  arXiv  Detail & Related papers  (2021-11-01T08:24:10Z)
• Model-based 3D Hand Reconstruction via Self-Supervised Learning [72.0817813032385]
  Reconstructing a 3D hand from a single-view RGB image is challenging due to various hand configurations and depth ambiguity. We propose S2HAND, a self-supervised 3D hand reconstruction network that can jointly estimate pose, shape, texture, and the camera viewpoint. For the first time, we demonstrate the feasibility of training an accurate 3D hand reconstruction network without relying on manual annotations.
  arXiv  Detail & Related papers  (2021-03-22T10:12:43Z)
• MM-Hand: 3D-Aware Multi-Modal Guided Hand Generative Network for 3D Hand Pose Synthesis [81.40640219844197]
  Estimating the 3D hand pose from a monocular RGB image is important but challenging. A solution is training on large-scale RGB hand images with accurate 3D hand keypoint annotations. We have developed a learning-based approach to synthesize realistic, diverse, and 3D pose-preserving hand images.
  arXiv  Detail & Related papers  (2020-10-02T18:27:34Z)
• Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wild [59.158592526006814]
  We train our network by gathering a large-scale dataset of hand action in YouTube videos. Our weakly-supervised mesh convolutions-based system largely outperforms state-of-the-art methods, even halving the errors on the in the wild benchmark.
  arXiv  Detail & Related papers  (2020-04-04T14:35:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.