Hand-Shadow Poser

• URL: http://arxiv.org/abs/2505.07012v1
• Date: Sun, 11 May 2025 15:15:35 GMT
• Title: Hand-Shadow Poser
• Authors: Hao Xu, Yinqiao Wang, Niloy J. Mitra, Shuaicheng Liu, Pheng-Ann Heng, Chi-Wing Fu,
• Abstract summary: Hand shadow art is a captivating art form, creatively using hand shadows to reproduce expressive shapes on the wall.<n>In this work, we study an inverse problem: given a target shape, find the poses of left and right hands that together produce a shadow resembling the input.<n>To meet these challenges, we design Hand-Shadow Poser, a three-stage pipeline, to decouple the anatomical constraints (by hand) and semantic constraints (by shadow shape)
• Score: 119.01376087903874
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Hand shadow art is a captivating art form, creatively using hand shadows to reproduce expressive shapes on the wall. In this work, we study an inverse problem: given a target shape, find the poses of left and right hands that together best produce a shadow resembling the input. This problem is nontrivial, since the design space of 3D hand poses is huge while being restrictive due to anatomical constraints. Also, we need to attend to the input's shape and crucial features, though the input is colorless and textureless. To meet these challenges, we design Hand-Shadow Poser, a three-stage pipeline, to decouple the anatomical constraints (by hand) and semantic constraints (by shadow shape): (i) a generative hand assignment module to explore diverse but reasonable left/right-hand shape hypotheses; (ii) a generalized hand-shadow alignment module to infer coarse hand poses with a similarity-driven strategy for selecting hypotheses; and (iii) a shadow-feature-aware refinement module to optimize the hand poses for physical plausibility and shadow feature preservation. Further, we design our pipeline to be trainable on generic public hand data, thus avoiding the need for any specialized training dataset. For method validation, we build a benchmark of 210 diverse shadow shapes of varying complexity and a comprehensive set of metrics, including a novel DINOv2-based evaluation metric. Through extensive comparisons with multiple baselines and user studies, our approach is demonstrated to effectively generate bimanual hand poses for a large variety of hand shapes for over 85% of the benchmark cases.

Related papers

• HaSPeR: An Image Repository for Hand Shadow Puppet Recognition [2.048226951354646]
  Hand shadow puppetry, also known as shadowgraphy or ombromanie, is a form of theatrical art and storytelling.<n>We introduce a novel dataset consisting of 15,000 images of hand shadow puppets across 15 classes extracted from both professional and amateur hand shadow puppeteer clips.<n>Our findings show a substantial performance superiority of skip-connected convolutional models over attention-based transformer architectures.
  arXiv  Detail & Related papers  (2024-08-19T18:56:24Z)
• HandNeRF: Neural Radiance Fields for Animatable Interacting Hands [122.32855646927013]
  We propose a novel framework to reconstruct accurate appearance and geometry with neural radiance fields (NeRF) for interacting hands. We conduct extensive experiments to verify the merits of our proposed HandNeRF and report a series of state-of-the-art results.
  arXiv  Detail & Related papers  (2023-03-24T06:19:19Z)
• Im2Hands: Learning Attentive Implicit Representation of Interacting Two-Hand Shapes [58.551154822792284]
  Implicit Two Hands (Im2Hands) is the first neural implicit representation of two interacting hands. Im2Hands can produce fine-grained geometry of two hands with high hand-to-hand and hand-to-image coherency. We experimentally demonstrate the effectiveness of Im2Hands on two-hand reconstruction in comparison to related methods.
  arXiv  Detail & Related papers  (2023-02-28T06:38:25Z)
• Mask2Hand: Learning to Predict the 3D Hand Pose and Shape from Shadow [13.9320397231143]
  Mask2Hand learns to solve the challenging task of predicting 3D hand pose and shape from a 2D binary mask of hand silhouette/shadow without additional manually-annotated data. Experiments show that our method, which takes a single binary mask as the input, can achieve comparable prediction accuracy on both unaligned and aligned settings.
  arXiv  Detail & Related papers  (2022-05-31T06:04:27Z)
• Collaborative Learning for Hand and Object Reconstruction with Attention-guided Graph Convolution [49.10497573378427]
  Estimating the pose and shape of hands and objects under interaction finds numerous applications including augmented and virtual reality. Our algorithm is optimisation to object models, and it learns the physical rules governing hand-object interaction. Experiments using four widely-used benchmarks show that our framework achieves beyond state-of-the-art accuracy in 3D pose estimation, as well as recovers dense 3D hand and object shapes.
  arXiv  Detail & Related papers  (2022-04-27T17:00:54Z)
• LISA: Learning Implicit Shape and Appearance of Hands [33.477530463242275]
  This paper proposes a do-it-all neural model of human hands, named LISA. The model can capture accurate hand shape and appearance, generalize to arbitrary hand subjects, provide dense surface correspondences, be reconstructed from images in the wild and easily animated.
  arXiv  Detail & Related papers  (2022-04-04T17:59:03Z)
• HandVoxNet: Deep Voxel-Based Network for 3D Hand Shape and Pose Estimation from a Single Depth Map [72.93634777578336]
  We propose a novel architecture with 3D convolutions trained in a weakly-supervised manner. The proposed approach improves over the state of the art by 47.8% on the SynHand5M dataset. Our method produces visually more reasonable and realistic hand shapes on NYU and BigHand2.2M datasets.
  arXiv  Detail & Related papers  (2020-04-03T14:27:16Z)
• Measuring Generalisation to Unseen Viewpoints, Articulations, Shapes and Objects for 3D Hand Pose Estimation under Hand-Object Interaction [137.28465645405655]
  HANDS'19 is a challenge to evaluate the abilities of current 3D hand pose estimators (HPEs) to interpolate and extrapolate the poses of a training set. We show that the accuracy of state-of-the-art methods can drop, and that they fail mostly on poses absent from the training set.
  arXiv  Detail & Related papers  (2020-03-30T19:28:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.