Pose-aware Attention Network for Flexible Motion Retargeting by Body Part

• URL: http://arxiv.org/abs/2306.08006v1
• Date: Tue, 13 Jun 2023 08:49:29 GMT
• Title: Pose-aware Attention Network for Flexible Motion Retargeting by Body Part
• Authors: Lei Hu, Zihao Zhang, Chongyang Zhong, Boyuan Jiang, Shihong Xia
• Abstract summary: Motion is a fundamental problem in computer graphics and computer vision. Existing approaches usually have many strict requirements. We propose a novel, flexible motion framework.
• Score: 17.637846838499737
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Motion retargeting is a fundamental problem in computer graphics and computer vision. Existing approaches usually have many strict requirements, such as the source-target skeletons needing to have the same number of joints or share the same topology. To tackle this problem, we note that skeletons with different structure may have some common body parts despite the differences in joint numbers. Following this observation, we propose a novel, flexible motion retargeting framework. The key idea of our method is to regard the body part as the basic retargeting unit rather than directly retargeting the whole body motion. To enhance the spatial modeling capability of the motion encoder, we introduce a pose-aware attention network (PAN) in the motion encoding phase. The PAN is pose-aware since it can dynamically predict the joint weights within each body part based on the input pose, and then construct a shared latent space for each body part by feature pooling. Extensive experiments show that our approach can generate better motion retargeting results both qualitatively and quantitatively than state-of-the-art methods. Moreover, we also show that our framework can generate reasonable results even for a more challenging retargeting scenario, like retargeting between bipedal and quadrupedal skeletons because of the body part retargeting strategy and PAN. Our code is publicly available.

Related papers

• Joint-Motion Mutual Learning for Pose Estimation in Videos [21.77871402339573]
  Human pose estimation in videos has long been a compelling yet challenging task within the realm of computer vision. Recent methods strive to integrate multi-frame visual features generated by a backbone network for pose estimation. We propose a novel joint-motion mutual learning framework for pose estimation.
  arXiv  Detail & Related papers  (2024-08-05T07:37:55Z)
• Part-aware Prototypical Graph Network for One-shot Skeleton-based Action Recognition [57.86960990337986]
  One-shot skeleton-based action recognition poses unique challenges in learning transferable representation from base classes to novel classes. We propose a part-aware prototypical representation for one-shot skeleton-based action recognition. We demonstrate the effectiveness of our method on two public skeleton-based action recognition datasets.
  arXiv  Detail & Related papers  (2022-08-19T04:54:56Z)
• SpatioTemporal Focus for Skeleton-based Action Recognition [66.8571926307011]
  Graph convolutional networks (GCNs) are widely adopted in skeleton-based action recognition. We argue that the performance of recent proposed skeleton-based action recognition methods is limited by the following factors. Inspired by the recent attention mechanism, we propose a multi-grain contextual focus module, termed MCF, to capture the action associated relation information.
  arXiv  Detail & Related papers  (2022-03-31T02:45:24Z)
• Learning Target-aware Representation for Visual Tracking via Informative Interactions [49.552877881662475]
  We introduce a novel backbone architecture to improve target-perception ability of feature representation for tracking. The proposed GIM module and InBN mechanism are general and applicable to different backbone types including CNN and Transformer.
  arXiv  Detail & Related papers  (2022-01-07T16:22:27Z)
• Hierarchical Neural Implicit Pose Network for Animation and Motion Retargeting [66.69067601079706]
  HIPNet is a neural implicit pose network trained on multiple subjects across many poses. We employ a hierarchical skeleton-based representation to learn a signed distance function on a canonical unposed space. We achieve state-of-the-art results on various single-subject and multi-subject benchmarks.
  arXiv  Detail & Related papers  (2021-12-02T03:25:46Z)
• Learning Local Recurrent Models for Human Mesh Recovery [50.85467243778406]
  We present a new method for video mesh recovery that divides the human mesh into several local parts following the standard skeletal model. We then model the dynamics of each local part with separate recurrent models, with each model conditioned appropriately based on the known kinematic structure of the human body. This results in a structure-informed local recurrent learning architecture that can be trained in an end-to-end fashion with available annotations.
  arXiv  Detail & Related papers  (2021-07-27T14:30:33Z)
• Pose-based Modular Network for Human-Object Interaction Detection [5.6397911482914385]
  We contribute a Pose-based Modular Network (PMN) which explores the absolute pose features and relative spatial pose features to improve HOI detection. To evaluate our proposed method, we combine the module with the state-of-the-art model named VS-GATs and obtain significant improvement on two public benchmarks.
  arXiv  Detail & Related papers  (2020-08-05T10:56:09Z)
• Skeleton-Aware Networks for Deep Motion Retargeting [83.65593033474384]
  We introduce a novel deep learning framework for data-driven motion between skeletons. Our approach learns how to retarget without requiring any explicit pairing between the motions in the training set.
  arXiv  Detail & Related papers  (2020-05-12T12:51:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.