Animating the Uncaptured: Humanoid Mesh Animation with Video Diffusion Models

• URL: http://arxiv.org/abs/2503.15996v1
• Date: Thu, 20 Mar 2025 10:00:22 GMT
• Title: Animating the Uncaptured: Humanoid Mesh Animation with Video Diffusion Models
• Authors: Marc Benedí San Millán, Angela Dai, Matthias Nießner,
• Abstract summary: Animation of humanoid characters is essential in various graphics applications.<n>We propose an approach to synthesize 4D animated sequences of input static 3D humanoid meshes.
• Score: 71.78723353724493
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Animation of humanoid characters is essential in various graphics applications, but requires significant time and cost to create realistic animations. We propose an approach to synthesize 4D animated sequences of input static 3D humanoid meshes, leveraging strong generalized motion priors from generative video models -- as such video models contain powerful motion information covering a wide variety of human motions. From an input static 3D humanoid mesh and a text prompt describing the desired animation, we synthesize a corresponding video conditioned on a rendered image of the 3D mesh. We then employ an underlying SMPL representation to animate the corresponding 3D mesh according to the video-generated motion, based on our motion optimization. This enables a cost-effective and accessible solution to enable the synthesis of diverse and realistic 4D animations.

Related papers

• PhysAnimator: Physics-Guided Generative Cartoon Animation [19.124321553546242]
  PhysAnimator is a novel approach for generating anime-stylized animation from static anime illustrations. To capture the fluidity and exaggeration characteristic of anime, we perform image-space deformable body simulations on extracted mesh geometries. We extract and warp sketches from the simulation sequence, generating a texture-agnostic representation, and employ a sketch-guided video diffusion model to synthesize high-quality animation frames.
  arXiv  Detail & Related papers  (2025-01-27T22:48:36Z)
• Deblur-Avatar: Animatable Avatars from Motion-Blurred Monocular Videos [64.10307207290039]
  We introduce a novel framework for modeling high-fidelity, animatable 3D human avatars from motion-blurred monocular video inputs.<n>By explicitly modeling human motion trajectories during exposure time, we jointly optimize the trajectories and 3D Gaussians to reconstruct sharp, high-quality human avatars.
  arXiv  Detail & Related papers  (2025-01-23T02:31:57Z)
• Gaussians-to-Life: Text-Driven Animation of 3D Gaussian Splatting Scenes [49.26872036160368]
  We propose a method for animating parts of high-quality 3D scenes in a Gaussian Splatting representation.<n>We find that, in contrast to prior work, this enables realistic animations of complex, pre-existing 3D scenes.
  arXiv  Detail & Related papers  (2024-11-28T16:01:58Z)
• Make-It-Animatable: An Efficient Framework for Authoring Animation-Ready 3D Characters [86.13319549186959]
  We present Make-It-Animatable, a novel data-driven method to make any 3D humanoid model ready for character animation in less than one second.<n>Our framework generates high-quality blend weights, bones, and pose transformations.<n>Compared to existing methods, our approach demonstrates significant improvements in both quality and speed.
  arXiv  Detail & Related papers  (2024-11-27T10:18:06Z)
• MotionDreamer: Exploring Semantic Video Diffusion features for Zero-Shot 3D Mesh Animation [10.263762787854862]
  We propose a technique for automatic re-animation of various 3D shapes based on a motion prior extracted from a video diffusion model. We leverage an explicit mesh-based representation compatible with existing computer-graphics pipelines. Our time-efficient zero-shot method achieves a superior performance re-animating a diverse set of 3D shapes.
  arXiv  Detail & Related papers  (2024-05-30T15:30:38Z)
• MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Model [74.84435399451573]
  This paper studies the human image animation task, which aims to generate a video of a certain reference identity following a particular motion sequence. Existing animation works typically employ the frame-warping technique to animate the reference image towards the target motion. We introduce MagicAnimate, a diffusion-based framework that aims at enhancing temporal consistency, preserving reference image faithfully, and improving animation fidelity.
  arXiv  Detail & Related papers  (2023-11-27T18:32:31Z)
• QS-Craft: Learning to Quantize, Scrabble and Craft for Conditional Human Motion Animation [66.97112599818507]
  This paper studies the task of conditional Human Motion Animation (cHMA) Given a source image and a driving video, the model should animate the new frame sequence. The key novelties come from the newly introduced three key steps: quantize, scrabble and craft.
  arXiv  Detail & Related papers  (2022-03-22T11:34:40Z)
• Action2video: Generating Videos of Human 3D Actions [31.665831044217363]
  We aim to tackle the interesting yet challenging problem of generating videos of diverse and natural human motions from prescribed action categories. Key issue lies in the ability to synthesize multiple distinct motion sequences that are realistic in their visual appearances. Action2motionally generates plausible 3D pose sequences of a prescribed action category, which are processed and rendered by motion2video to form 2D videos.
  arXiv  Detail & Related papers  (2021-11-12T20:20:37Z)
• Going beyond Free Viewpoint: Creating Animatable Volumetric Video of Human Performances [7.7824496657259665]
  We present an end-to-end pipeline for the creation of high-quality animatable volumetric video content of human performances. Semantic enrichment and geometric animation ability are achieved by establishing temporal consistency in the 3D data. For pose editing, we exploit the captured data as much as possible and kinematically deform the captured frames to fit a desired pose.
  arXiv  Detail & Related papers  (2020-09-02T09:46:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.