ToMiE: Towards Modular Growth in Enhanced SMPL Skeleton for 3D Human with Animatable Garments

• URL: http://arxiv.org/abs/2410.08082v1
• Date: Thu, 10 Oct 2024 16:25:52 GMT
• Title: ToMiE: Towards Modular Growth in Enhanced SMPL Skeleton for 3D Human with Animatable Garments
• Authors: Yifan Zhan, Qingtian Zhu, Muyao Niu, Mingze Ma, Jiancheng Zhao, Zhihang Zhong, Xiao Sun, Yu Qiao, Yinqiang Zheng,
• Abstract summary: We propose a modular growth strategy that enables the joint tree of the skeleton to expand adaptively. Specifically, our method, called ToMiE, consists of parent joints localization and external joints optimization. ToMiE manages to outperform other methods across various cases with garments, not only in rendering quality but also by offering free animation of grown joints.
• Score: 41.23897822168498
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we highlight a critical yet often overlooked factor in most 3D human tasks, namely modeling humans with complex garments. It is known that the parameterized formulation of SMPL is able to fit human skin; while complex garments, e.g., hand-held objects and loose-fitting garments, are difficult to get modeled within the unified framework, since their movements are usually decoupled with the human body. To enhance the capability of SMPL skeleton in response to this situation, we propose a modular growth strategy that enables the joint tree of the skeleton to expand adaptively. Specifically, our method, called ToMiE, consists of parent joints localization and external joints optimization. For parent joints localization, we employ a gradient-based approach guided by both LBS blending weights and motion kernels. Once the external joints are obtained, we proceed to optimize their transformations in SE(3) across different frames, enabling rendering and explicit animation. ToMiE manages to outperform other methods across various cases with garments, not only in rendering quality but also by offering free animation of grown joints, thereby enhancing the expressive ability of SMPL skeleton for a broader range of applications.

Related papers

• ASMR: Adaptive Skeleton-Mesh Rigging and Skinning via 2D Generative Prior [5.429282997550316]
  We present a novel method for the automatic rigging and skinning of character meshes using skeletal motion data. The proposed method predicts the optimal skeleton aligned with the size and proportion of the mesh as well as defines skinning weights for various mesh-skeleton configurations.
  arXiv  Detail & Related papers  (2025-03-17T15:59:02Z)
• MagicArticulate: Make Your 3D Models Articulation-Ready [109.35703811628045]
  We present MagicArticulate, an effective framework that automatically transforms static 3D models into articulation-ready assets. Our key contributions are threefold. First, we introduce Articulation-averse benchmark containing over 33k 3D models with high-quality articulation annotations, carefully curated from XL-XL. Extensive experiments demonstrate that MagicArticulate significantly outperforms existing methods across diverse object categories.
  arXiv  Detail & Related papers  (2025-02-17T18:53:27Z)
• JADE: Joint-aware Latent Diffusion for 3D Human Generative Modeling [62.77347895550087]
  We introduce JADE, a generative framework that learns the variations of human shapes with fined-grained control. Our key insight is a joint-aware latent representation that decomposes human bodies into skeleton structures. To generate coherent and plausible human shapes under our proposed decomposition, we also present a cascaded pipeline.
  arXiv  Detail & Related papers  (2024-12-29T14:18:35Z)
• Multi-Resolution Generative Modeling of Human Motion from Limited Data [3.5229503563299915]
  We present a generative model that learns to synthesize human motion from limited training sequences. The model adeptly captures human motion patterns by integrating skeletal convolution layers and a multi-scale architecture.
  arXiv  Detail & Related papers  (2024-11-25T15:36:29Z)
• Motion-Oriented Compositional Neural Radiance Fields for Monocular Dynamic Human Modeling [10.914612535745789]
  This paper introduces Motion-oriented Compositional Neural Radiance Fields (MoCo-NeRF) MoCo-NeRF is a framework designed to perform free-viewpoint rendering of monocular human videos.
  arXiv  Detail & Related papers  (2024-07-16T17:59:01Z)
• InterControl: Zero-shot Human Interaction Generation by Controlling Every Joint [67.6297384588837]
  We introduce a novel controllable motion generation method, InterControl, to encourage the synthesized motions maintaining the desired distance between joint pairs. We demonstrate that the distance between joint pairs for human-wise interactions can be generated using an off-the-shelf Large Language Model.
  arXiv  Detail & Related papers  (2023-11-27T14:32:33Z)
• Double-chain Constraints for 3D Human Pose Estimation in Images and Videos [21.42410292863492]
  Reconstructing 3D poses from 2D poses lacking depth information is challenging due to the complexity and diversity of human motion. We propose a novel model, called Double-chain Graph Convolutional Transformer (DC-GCT), to constrain the pose. We show that DC-GCT achieves state-of-the-art performance on two challenging datasets.
  arXiv  Detail & Related papers  (2023-08-10T02:41:18Z)
• Generalizable Neural Performer: Learning Robust Radiance Fields for Human Novel View Synthesis [52.720314035084215]
  This work targets at using a general deep learning framework to synthesize free-viewpoint images of arbitrary human performers. We present a simple yet powerful framework, named Generalizable Neural Performer (GNR), that learns a generalizable and robust neural body representation. Experiments on GeneBody-1.0 and ZJU-Mocap show better robustness of our methods than recent state-of-the-art generalizable methods.
  arXiv  Detail & Related papers  (2022-04-25T17:14:22Z)
• SM-SGE: A Self-Supervised Multi-Scale Skeleton Graph Encoding Framework for Person Re-Identification [12.347724797202865]
  Person re-identification via 3D skeletons is an emerging topic with great potential in security-critical applications. Existing methods typically learn body and motion features from the body-joint trajectory. We propose a Self-supervised Multi-scale Graph (SM-SGE) framework that comprehensively models human body, component relations, and skeleton dynamics.
  arXiv  Detail & Related papers  (2021-07-05T09:53:08Z)
• Revisiting Skeleton-based Action Recognition [107.08112310075114]
  PoseC3D is a new approach to skeleton-based action recognition, which relies on a 3D heatmap instead stack a graph sequence as the base representation of human skeletons. On four challenging datasets, PoseC3D consistently obtains superior performance, when used alone on skeletons and in combination with the RGB modality.
  arXiv  Detail & Related papers  (2021-04-28T06:32:17Z)
• SMPLicit: Topology-aware Generative Model for Clothed People [65.84665248796615]
  We introduce SMPLicit, a novel generative model to jointly represent body pose, shape and clothing geometry. In the experimental section, we demonstrate SMPLicit can be readily used for fitting 3D scans and for 3D reconstruction in images of dressed people.
  arXiv  Detail & Related papers  (2021-03-11T18:57:03Z)
• STAR: Sparse Trained Articulated Human Body Regressor [62.71047277943326]
  We introduce STAR, which is quantitatively and qualitatively superior to SMPL. SMPL has a huge number of parameters resulting from its use of global blend shapes. SMPL factors pose-dependent deformations from body shape while, in reality, people with different shapes deform differently. We show that the shape space of SMPL is not rich enough to capture the variation in the human population.
  arXiv  Detail & Related papers  (2020-08-19T16:27:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.