ToMiE: Towards Explicit Exoskeleton for the Reconstruction of Complicated 3D Human Avatars

• URL: http://arxiv.org/abs/2410.08082v2
• Date: Thu, 26 Jun 2025 06:47:51 GMT
• Title: ToMiE: Towards Explicit Exoskeleton for the Reconstruction of Complicated 3D Human Avatars
• Authors: Yifan Zhan, Qingtian Zhu, Muyao Niu, Mingze Ma, Jiancheng Zhao, Zhihang Zhong, Xiao Sun, Yu Qiao, Yinqiang Zheng,
• Abstract summary: We propose a growth strategy that enables the joint tree of the skeleton to expand adaptively.<n>Specifically, our method, called ToMiE, consists of parent joints localization and external joints optimization.<n>ToMiE manages to outperform other methods across various cases with hand-held objects and loose-fitting clothing, not only in rendering quality but also by offering free animation of grown joints.
• Score: 41.23897822168498
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we highlight a critical yet often overlooked factor in most 3D human tasks, namely modeling complicated 3D human with with hand-held objects or loose-fitting clothing. It is known that the parameterized formulation of SMPL is able to fit human skin; while hand-held objects and loose-fitting clothing, are difficult to get modeled within the unified framework, since their movements are usually decoupled with the human body. To enhance the capability of SMPL skeleton in response to this situation, we propose a growth strategy that enables the joint tree of the skeleton to expand adaptively. Specifically, our method, called ToMiE, consists of parent joints localization and external joints optimization. For parent joints localization, we employ a gradient-based approach guided by both LBS blending weights and motion kernels. Once the external joints are obtained, we proceed to optimize their transformations in SE(3) across different frames, enabling rendering and explicit animation. ToMiE manages to outperform other methods across various cases with hand-held objects and loose-fitting clothing, not only in rendering quality but also by offering free animation of grown joints, thereby enhancing the expressive ability of SMPL skeleton for a broader range of applications.

Related papers

• ArtiLatent: Realistic Articulated 3D Object Generation via Structured Latents [31.495577251319315]
  ArtiLatent is a generative framework that synthesizes human-made 3D objects with fine-grained geometry, accurate articulation, and realistic appearance.
  arXiv  Detail & Related papers  (2025-10-24T13:08:15Z)
• From Skin to Skeleton: Towards Biomechanically Accurate 3D Digital Humans [50.014530130312714]
  We develop SKEL, which re-rigs the SMPL body model with a biomechanics skeleton.<n>We show that SKEL has more biomechanically accurate joint locations than SMPL, and the bones fit inside the body surface better than previous methods.
  arXiv  Detail & Related papers  (2025-09-08T12:24:27Z)
• ATLAS: Decoupling Skeletal and Shape Parameters for Expressive Parametric Human Modeling [43.66748605071065]
  We present ATLAS, a high-fidelity body model learned from 600k high-resolution scans captured using 240 synchronized cameras.<n>We explicitly decouple the shape and skeleton bases by grounding our mesh representation in the human skeleton.<n> ATLAS outperforms existing methods by fitting unseen subjects in diverse poses more accurately, and quantitative evaluations show that our non-linear pose correctives more effectively capture complex poses compared to linear models.
  arXiv  Detail & Related papers  (2025-08-21T17:58:56Z)
• Puppeteer: Rig and Animate Your 3D Models [105.11046762553121]
  Puppeteer is a comprehensive framework that addresses both automatic rigging and animation for diverse 3D objects.<n>Our system first predicts plausible skeletal structures via an auto-regressive transformer.<n>It then infers skinning weights via an attention-based architecture.
  arXiv  Detail & Related papers  (2025-08-14T17:59:31Z)
• PoseGRAF: Geometric-Reinforced Adaptive Fusion for Monocular 3D Human Pose Estimation [5.223657684081615]
  Existing monocular 3D pose estimation methods rely on joint positional features, while overlooking intrinsic directional and angular correlations within the skeleton.<n>We propose the PoseGRAF framework to address these challenges.<n> Experimental results on the Human3.6M and MPI-INF-3DHP datasets show that our method exceeds state-of-the-art approaches.
  arXiv  Detail & Related papers  (2025-06-17T14:59:56Z)
• HumanRAM: Feed-forward Human Reconstruction and Animation Model using Transformers [60.86393841247567]
  HumanRAM is a novel feed-forward approach for generalizable human reconstruction and animation from monocular or sparse human images.<n>Our approach integrates human reconstruction and animation into a unified framework by introducing explicit pose conditions.<n> Experiments show that HumanRAM significantly surpasses previous methods in terms of reconstruction accuracy, animation fidelity, and generalization performance on real-world datasets.
  arXiv  Detail & Related papers  (2025-06-03T17:50:05Z)
• Controllable Human-centric Keyframe Interpolation with Generative Prior [55.16558476905587]
  We introduce PoseFuse3D Keyframe Interpolator (PoseFuse3D-KI), a novel framework that integrates 3D human guidance signals into the diffusion process.<n>We show that PoseFuse3D-KI consistently outperforms state-of-the-art baselines on CHKI-Video.
  arXiv  Detail & Related papers  (2025-06-03T17:50:05Z)
• ASMR: Adaptive Skeleton-Mesh Rigging and Skinning via 2D Generative Prior [5.429282997550316]
  We present a novel method for the automatic rigging and skinning of character meshes using skeletal motion data. The proposed method predicts the optimal skeleton aligned with the size and proportion of the mesh as well as defines skinning weights for various mesh-skeleton configurations.
  arXiv  Detail & Related papers  (2025-03-17T15:59:02Z)
• MagicArticulate: Make Your 3D Models Articulation-Ready [109.35703811628045]
  We present MagicArticulate, an effective framework that automatically transforms static 3D models into articulation-ready assets. Our key contributions are threefold. First, we introduce Articulation-averse benchmark containing over 33k 3D models with high-quality articulation annotations, carefully curated from XL-XL. Extensive experiments demonstrate that MagicArticulate significantly outperforms existing methods across diverse object categories.
  arXiv  Detail & Related papers  (2025-02-17T18:53:27Z)
• JADE: Joint-aware Latent Diffusion for 3D Human Generative Modeling [62.77347895550087]
  We introduce JADE, a generative framework that learns the variations of human shapes with fined-grained control. Our key insight is a joint-aware latent representation that decomposes human bodies into skeleton structures. To generate coherent and plausible human shapes under our proposed decomposition, we also present a cascaded pipeline.
  arXiv  Detail & Related papers  (2024-12-29T14:18:35Z)
• Multi-Resolution Generative Modeling of Human Motion from Limited Data [3.5229503563299915]
  We present a generative model that learns to synthesize human motion from limited training sequences. The model adeptly captures human motion patterns by integrating skeletal convolution layers and a multi-scale architecture.
  arXiv  Detail & Related papers  (2024-11-25T15:36:29Z)
• GST: Precise 3D Human Body from a Single Image with Gaussian Splatting Transformers [23.96688843662126]
  Reconstructing posed 3D human models from monocular images has important applications in the sports industry.<n>We combine 3D human pose and shape estimation with 3D Gaussian Splatting (3DGS), a representation of the scene composed of a mixture of Gaussians.<n>We show that this combination can achieve near real-time inference of 3D human models from a single image without expensive diffusion models or 3D points supervision.
  arXiv  Detail & Related papers  (2024-09-06T11:34:24Z)
• Motion-Oriented Compositional Neural Radiance Fields for Monocular Dynamic Human Modeling [10.914612535745789]
  This paper introduces Motion-oriented Compositional Neural Radiance Fields (MoCo-NeRF) MoCo-NeRF is a framework designed to perform free-viewpoint rendering of monocular human videos.
  arXiv  Detail & Related papers  (2024-07-16T17:59:01Z)
• Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance [25.346255905155424]
  We introduce a methodology for human image animation by leveraging a 3D human parametric model within a latent diffusion framework. By representing the 3D human parametric model as the motion guidance, we can perform parametric shape alignment of the human body between the reference image and the source video motion. Our approach also exhibits superior generalization capabilities on the proposed in-the-wild dataset.
  arXiv  Detail & Related papers  (2024-03-21T18:52:58Z)
• InterControl: Zero-shot Human Interaction Generation by Controlling Every Joint [67.6297384588837]
  We introduce a novel controllable motion generation method, InterControl, to encourage the synthesized motions maintaining the desired distance between joint pairs. We demonstrate that the distance between joint pairs for human-wise interactions can be generated using an off-the-shelf Large Language Model.
  arXiv  Detail & Related papers  (2023-11-27T14:32:33Z)
• Double-chain Constraints for 3D Human Pose Estimation in Images and Videos [21.42410292863492]
  Reconstructing 3D poses from 2D poses lacking depth information is challenging due to the complexity and diversity of human motion. We propose a novel model, called Double-chain Graph Convolutional Transformer (DC-GCT), to constrain the pose. We show that DC-GCT achieves state-of-the-art performance on two challenging datasets.
  arXiv  Detail & Related papers  (2023-08-10T02:41:18Z)
• Generalizable Neural Performer: Learning Robust Radiance Fields for Human Novel View Synthesis [52.720314035084215]
  This work targets at using a general deep learning framework to synthesize free-viewpoint images of arbitrary human performers. We present a simple yet powerful framework, named Generalizable Neural Performer (GNR), that learns a generalizable and robust neural body representation. Experiments on GeneBody-1.0 and ZJU-Mocap show better robustness of our methods than recent state-of-the-art generalizable methods.
  arXiv  Detail & Related papers  (2022-04-25T17:14:22Z)
• SM-SGE: A Self-Supervised Multi-Scale Skeleton Graph Encoding Framework for Person Re-Identification [12.347724797202865]
  Person re-identification via 3D skeletons is an emerging topic with great potential in security-critical applications. Existing methods typically learn body and motion features from the body-joint trajectory. We propose a Self-supervised Multi-scale Graph (SM-SGE) framework that comprehensively models human body, component relations, and skeleton dynamics.
  arXiv  Detail & Related papers  (2021-07-05T09:53:08Z)
• Revisiting Skeleton-based Action Recognition [107.08112310075114]
  PoseC3D is a new approach to skeleton-based action recognition, which relies on a 3D heatmap instead stack a graph sequence as the base representation of human skeletons. On four challenging datasets, PoseC3D consistently obtains superior performance, when used alone on skeletons and in combination with the RGB modality.
  arXiv  Detail & Related papers  (2021-04-28T06:32:17Z)
• SMPLicit: Topology-aware Generative Model for Clothed People [65.84665248796615]
  We introduce SMPLicit, a novel generative model to jointly represent body pose, shape and clothing geometry. In the experimental section, we demonstrate SMPLicit can be readily used for fitting 3D scans and for 3D reconstruction in images of dressed people.
  arXiv  Detail & Related papers  (2021-03-11T18:57:03Z)
• STAR: Sparse Trained Articulated Human Body Regressor [62.71047277943326]
  We introduce STAR, which is quantitatively and qualitatively superior to SMPL. SMPL has a huge number of parameters resulting from its use of global blend shapes. SMPL factors pose-dependent deformations from body shape while, in reality, people with different shapes deform differently. We show that the shape space of SMPL is not rich enough to capture the variation in the human population.
  arXiv  Detail & Related papers  (2020-08-19T16:27:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.