Two-Person Interaction Augmentation with Skeleton Priors

• URL: http://arxiv.org/abs/2404.05490v2
• Date: Tue, 9 Apr 2024 18:55:43 GMT
• Title: Two-Person Interaction Augmentation with Skeleton Priors
• Authors: Baiyi Li, Edmond S. L. Ho, Hubert P. H. Shum, He Wang,
• Abstract summary: We propose a new deep learning method for two-body skeletal interaction motion augmentation. Our system can learn effectively from a relatively small amount of data and generalize to drastically different skeleton sizes.
• Score: 16.65884142618145
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Close and continuous interaction with rich contacts is a crucial aspect of human activities (e.g. hugging, dancing) and of interest in many domains like activity recognition, motion prediction, character animation, etc. However, acquiring such skeletal motion is challenging. While direct motion capture is expensive and slow, motion editing/generation is also non-trivial, as complex contact patterns with topological and geometric constraints have to be retained. To this end, we propose a new deep learning method for two-body skeletal interaction motion augmentation, which can generate variations of contact-rich interactions with varying body sizes and proportions while retaining the key geometric/topological relations between two bodies. Our system can learn effectively from a relatively small amount of data and generalize to drastically different skeleton sizes. Through exhaustive evaluation and comparison, we show it can generate high-quality motions, has strong generalizability and outperforms traditional optimization-based methods and alternative deep learning solutions.

Related papers

• Joint-Motion Mutual Learning for Pose Estimation in Videos [21.77871402339573]
  Human pose estimation in videos has long been a compelling yet challenging task within the realm of computer vision. Recent methods strive to integrate multi-frame visual features generated by a backbone network for pose estimation. We propose a novel joint-motion mutual learning framework for pose estimation.
  arXiv  Detail & Related papers  (2024-08-05T07:37:55Z)
• GEARS: Local Geometry-aware Hand-object Interaction Synthesis [38.75942505771009]
  We introduce a novel joint-centered sensor designed to reason about local object geometry near potential interaction regions. As an important step towards mitigating the learning complexity, we transform the points from global frame to template hand frame and use a shared module to process sensor features of each individual joint. This is followed by a perceptual-temporal transformer network aimed at capturing correlation among the joints in different dimensions.
  arXiv  Detail & Related papers  (2024-04-02T09:18:52Z)
• ReMoS: 3D Motion-Conditioned Reaction Synthesis for Two-Person Interactions [66.87211993793807]
  We present ReMoS, a denoising diffusion based model that synthesizes full body motion of a person in two person interaction scenario. We demonstrate ReMoS across challenging two person scenarios such as pair dancing, Ninjutsu, kickboxing, and acrobatics. We also contribute the ReMoCap dataset for two person interactions containing full body and finger motions.
  arXiv  Detail & Related papers  (2023-11-28T18:59:52Z)
• InterControl: Zero-shot Human Interaction Generation by Controlling Every Joint [67.6297384588837]
  We introduce a novel controllable motion generation method, InterControl, to encourage the synthesized motions maintaining the desired distance between joint pairs. We demonstrate that the distance between joint pairs for human-wise interactions can be generated using an off-the-shelf Large Language Model.
  arXiv  Detail & Related papers  (2023-11-27T14:32:33Z)
• Persistent-Transient Duality: A Multi-mechanism Approach for Modeling Human-Object Interaction [58.67761673662716]
  Humans are highly adaptable, swiftly switching between different modes to handle different tasks, situations and contexts. In Human-object interaction (HOI) activities, these modes can be attributed to two mechanisms: (1) the large-scale consistent plan for the whole activity and (2) the small-scale children interactive actions that start and end along the timeline. This work proposes to model two concurrent mechanisms that jointly control human motion.
  arXiv  Detail & Related papers  (2023-07-24T12:21:33Z)
• Interaction Transformer for Human Reaction Generation [61.22481606720487]
  We propose a novel interaction Transformer (InterFormer) consisting of a Transformer network with both temporal and spatial attentions. Our method is general and can be used to generate more complex and long-term interactions.
  arXiv  Detail & Related papers  (2022-07-04T19:30:41Z)
• COUCH: Towards Controllable Human-Chair Interactions [44.66450508317131]
  We study the problem of synthesizing scene interactions conditioned on different contact positions on the object. We propose a novel synthesis framework COUCH that plans ahead the motion by predicting contact-aware control signals of the hands. Our method shows significant quantitative and qualitative improvements over existing methods for human-object interactions.
  arXiv  Detail & Related papers  (2022-05-01T19:14:22Z)
• Contact-Aware Retargeting of Skinned Motion [49.71236739408685]
  This paper introduces a motion estimation method that preserves self-contacts and prevents interpenetration. The method identifies self-contacts and ground contacts in the input motion, and optimize the motion to apply to the output skeleton. In experiments, our results quantitatively outperform previous methods and we conduct a user study where our retargeted motions are rated as higher-quality than those produced by recent works.
  arXiv  Detail & Related papers  (2021-09-15T17:05:02Z)
• Skeleton-Aware Networks for Deep Motion Retargeting [83.65593033474384]
  We introduce a novel deep learning framework for data-driven motion between skeletons. Our approach learns how to retarget without requiring any explicit pairing between the motions in the training set.
  arXiv  Detail & Related papers  (2020-05-12T12:51:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.