High-Fidelity and Long-Duration Human Image Animation with Diffusion Transformer

• URL: http://arxiv.org/abs/2512.21905v1
• Date: Fri, 26 Dec 2025 07:36:48 GMT
• Title: High-Fidelity and Long-Duration Human Image Animation with Diffusion Transformer
• Authors: Shen Zheng, Jiaran Cai, Yuansheng Guan, Shenneng Huang, Xingpei Ma, Junjie Cao, Hanfeng Zhao, Qiang Zhang, Shunsi Zhang, Xiao-Ping Zhang,
• Abstract summary: We propose a diffusion transformer (DiT)-based framework which focuses on generating high-fidelity and long-duration human animation videos.<n>First, we design a set of hybrid implicit guidance signals and a sharpness guidance factor, enabling our framework to additionally incorporate detailed facial and hand features as guidance.<n>Next, we incorporate the time-aware position shift fusion module, modify the input format within the DiT backbone, and refer to this mechanism as the Position Shift Adaptive Module.
• Score: 17.388852038062705
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Recent progress in diffusion models has significantly advanced the field of human image animation. While existing methods can generate temporally consistent results for short or regular motions, significant challenges remain, particularly in generating long-duration videos. Furthermore, the synthesis of fine-grained facial and hand details remains under-explored, limiting the applicability of current approaches in real-world, high-quality applications. To address these limitations, we propose a diffusion transformer (DiT)-based framework which focuses on generating high-fidelity and long-duration human animation videos. First, we design a set of hybrid implicit guidance signals and a sharpness guidance factor, enabling our framework to additionally incorporate detailed facial and hand features as guidance. Next, we incorporate the time-aware position shift fusion module, modify the input format within the DiT backbone, and refer to this mechanism as the Position Shift Adaptive Module, which enables video generation of arbitrary length. Finally, we introduce a novel data augmentation strategy and a skeleton alignment model to reduce the impact of human shape variations across different identities. Experimental results demonstrate that our method outperforms existing state-of-the-art approaches, achieving superior performance in both high-fidelity and long-duration human image animation.

Related papers

• EchoMotion: Unified Human Video and Motion Generation via Dual-Modality Diffusion Transformer [64.69014756863331]
  We introduce EchoMotion, a framework designed to model the joint distribution of appearance and human motion.<n>We also propose MVS-RoPE, which offers unified 3D positional encoding for both video and motion tokens.<n>Our findings reveal that explicitly representing human motion is to appearance, significantly boosting the coherence and plausibility of human-centric video generation.
  arXiv  Detail & Related papers  (2025-12-21T17:08:14Z)
• TalkingPose: Efficient Face and Gesture Animation with Feedback-guided Diffusion Model [18.910745982208965]
  TalkingPose is a novel diffusion-based framework for producing temporally consistent human upper-body animations.<n>We introduce a feedback-driven mechanism built upon image-based diffusion models to ensure continuous motion and enhance temporal coherence.<n>We also introduce a comprehensive, large-scale dataset to serve as a new benchmark for human upper-body animation.
  arXiv  Detail & Related papers  (2025-11-30T14:26:24Z)
• Stable Video-Driven Portraits [52.008400639227034]
  Animation aims to generate photo-realistic videos from a single source image by reenacting the expression and pose from a driving video.<n>Recent advances using diffusion models have demonstrated improved quality but remain constrained by weak control signals and architectural limitations.<n>We propose a novel diffusion based framework that leverages masked facial regions specifically the eyes, nose, and mouth from the driving video as strong motion control cues.
  arXiv  Detail & Related papers  (2025-09-22T08:11:08Z)
• HumanRAM: Feed-forward Human Reconstruction and Animation Model using Transformers [60.86393841247567]
  HumanRAM is a novel feed-forward approach for generalizable human reconstruction and animation from monocular or sparse human images.<n>Our approach integrates human reconstruction and animation into a unified framework by introducing explicit pose conditions.<n> Experiments show that HumanRAM significantly surpasses previous methods in terms of reconstruction accuracy, animation fidelity, and generalization performance on real-world datasets.
  arXiv  Detail & Related papers  (2025-06-03T17:50:05Z)
• HyperMotion: DiT-Based Pose-Guided Human Image Animation of Complex Motions [12.46263584777151]
  We introduce the textbfOpen-HyperMotionX dataset and textbfHyperMotionX Bench, which provide high-quality human pose annotations and curated video clips.<n>We also propose a simple yet powerful DiT-based video generation baseline and design spatial low-frequency enhanced RoPE.<n>Our method significantly improves structural stability and appearance consistency in highly dynamic human motion sequences.
  arXiv  Detail & Related papers  (2025-05-29T01:30:46Z)
• RAGME: Retrieval Augmented Video Generation for Enhanced Motion Realism [73.38167494118746]
  We propose a framework to improve the realism of motion in generated videos.<n>We advocate for the incorporation of a retrieval mechanism during the generation phase.<n>Our pipeline is designed to apply to any text-to-video diffusion model.
  arXiv  Detail & Related papers  (2025-04-09T08:14:05Z)
• DreamActor-M1: Holistic, Expressive and Robust Human Image Animation with Hybrid Guidance [9.898947423344884]
  We propose a diffusion transformer (DiT) based framework, DreamActor-M1, with hybrid guidance to overcome limitations.<n>For motion guidance, our hybrid control signals that integrate implicit facial representations, 3D head spheres, and 3D body skeletons achieve robust control of facial expressions and body movements.<n>Experiments demonstrate that our method outperforms the state-of-the-art works, delivering expressive results for portraits, upper-body, and full-body generation.
  arXiv  Detail & Related papers  (2025-04-02T13:30:32Z)
• EvAnimate: Event-conditioned Image-to-Video Generation for Human Animation [58.41979933166173]
  EvAnimate is the first method leveraging event streams as robust and precise motion cues for conditional human image animation.<n>High-quality and temporally coherent animations are achieved through a dual-branch architecture.<n>Experiment results show EvAnimate achieves high temporal fidelity and robust performance in scenarios where traditional video-derived cues fall short.
  arXiv  Detail & Related papers  (2025-03-24T11:05:41Z)
• Anchored Diffusion for Video Face Reenactment [17.343307538702238]
  We introduce Anchored Diffusion, a novel method for synthesizing relatively long and seamless videos. We train our model on video sequences with random non-uniform temporal spacing and incorporate temporal information via external guidance. During inference, we leverage the transformer architecture to modify the diffusion process, generating a batch of non-uniform sequences anchored to a common frame.
  arXiv  Detail & Related papers  (2024-07-21T13:14:17Z)
• UniAnimate: Taming Unified Video Diffusion Models for Consistent Human Image Animation [53.16986875759286]
  We present a UniAnimate framework to enable efficient and long-term human video generation. We map the reference image along with the posture guidance and noise video into a common feature space. We also propose a unified noise input that supports random noised input as well as first frame conditioned input.
  arXiv  Detail & Related papers  (2024-06-03T10:51:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.