Beyond Skeletons: Learning Animation Directly from Driving Videos with Same2X Training Strategy

• URL: http://arxiv.org/abs/2606.06903v1
• Date: Fri, 05 Jun 2026 04:39:46 GMT
• Title: Beyond Skeletons: Learning Animation Directly from Driving Videos with Same2X Training Strategy
• Abstract summary: We present DirectAnimator, a framework that bypasses pose extraction and directly learns from raw driving videos.<n>We introduce a Driving Cue Triplet consisting of pose, face, and location cues that captures motion, expression, and alignment in a semantically rich yet stable form.<n>We devise a Same2X training strategy that aligns cross-ID features with those learned from same-ID data, regularizing optimization and accelerating convergence.
• Score: 67.1159444608631
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Human image animation aims to generate a video from a static reference image, guided by pose information extracted from a driving video. Existing approaches often rely on pose estimators to extract intermediate representations, but such signals are prone to errors under occlusion or complex poses. Building on these observations, we present DirectAnimator, a framework that bypasses pose extraction and directly learns from raw driving videos. We introduce a Driving Cue Triplet consisting of pose, face, and location cues that captures motion, expression, and alignment in a semantically rich yet stable form, and we fuse them through a CueFusion DiT block for reliable control during denoising. To make learning dependable when the driving and reference identities differ, we devise a Same2X training strategy that aligns cross-ID features with those learned from same-ID data, regularizing optimization and accelerating convergence. Extensive experiments demonstrate that DirectAnimator attains state-of-the-art visual quality and identity preservation while remaining robust to occlusions and complex articulation, and it does so with fewer computational resources. Our project page is at https://directanimator.github.io/.

Related papers

• Wan-Animate-2: Pushing the Application Boundaries of Character Animation [26.88580952829003]
  Wan-Animate-2 is an end-to-end character animation framework that directly consumes the driving video within a redesigned Diffusion Transformer.<n>Our architecture achieves superior motion fidelity and identity preservation by eliminating intermediate motion extractors entirely.<n>We present Wan-Animate-2-Lite, an efficient variant that reduces inference latency to real-time thresholds.
  arXiv  Detail & Related papers  (2026-08-06T13:13:10Z)
• STARCaster: Spatio-Temporal AutoRegressive Video Diffusion for Identity- and View-Aware Talking Portraits [44.82339975771063]
  STARCaster is an identity-aware video diffusion model that addresses both speech-driven portrait animation and free-point-view talking portrait.<n>The model learns from longer temporal contexts than those generated at inference, mitigating the overly static animations common in existing autoregressive approaches.
  arXiv  Detail & Related papers  (2025-12-15T11:59:01Z)
• FactorPortrait: Controllable Portrait Animation via Disentangled Expression, Pose, and Viewpoint [49.80464592726769]
  We introduce FactorPortrait, a video diffusion method for controllable portrait animation.<n>Our method animates the portrait by transferring facial expressions and head movements from the driving video.<n>Our method outperforms existing approaches in realism, expressiveness, control accuracy, and view consistency.
  arXiv  Detail & Related papers  (2025-12-12T15:22:52Z)
• X-NeMo: Expressive Neural Motion Reenactment via Disentangled Latent Attention [52.94097577075215]
  X-NeMo is a zero-shot diffusion-based portrait animation pipeline.<n>It animates a static portrait using facial movements from a driving video of a different individual.
  arXiv  Detail & Related papers  (2025-07-30T22:46:52Z)
• StableAnimator++: Overcoming Pose Misalignment and Face Distortion for Human Image Animation [98.10527466949338]
  Current diffusion models for human image animation often struggle to maintain identity consistency.<n>We introduce StableAnimator++, the first ID-preserving video diffusion framework with learnable pose alignment.<n>We show how StableAnimator++ generates high-quality videos conditioned on a reference image and a pose sequence without any post-processing.
  arXiv  Detail & Related papers  (2025-07-20T17:59:26Z)
• MoSiC: Optimal-Transport Motion Trajectory for Dense Self-Supervised Learning [66.53533434848369]
  We propose a motion-guided self-learning framework that learns densely consistent representations.<n>We improve state-of-the-art by 1% to 6% on six image and video datasets and four evaluation benchmarks.
  arXiv  Detail & Related papers  (2025-06-10T11:20:32Z)
• MegActor: Harness the Power of Raw Video for Vivid Portrait Animation [16.013989935948114]
  MegActor harnesses the power of the raw videos for vivid portrait animation. We introduce a synthetic data generation framework for creating videos with consistent motion and expressions but inconsistent IDs. We further style transfer the appearance of the reference image to the driving video to eliminate the influence of facial details in the driving videos.
  arXiv  Detail & Related papers  (2024-05-31T14:33:13Z)
• ASCNet: Self-supervised Video Representation Learning with Appearance-Speed Consistency [62.38914747727636]
  We study self-supervised video representation learning, which is a challenging task due to 1) a lack of labels for explicit supervision and 2) unstructured and noisy visual information. Existing methods mainly use contrastive loss with video clips as the instances and learn visual representation by discriminating instances from each other. In this paper, we observe that the consistency between positive samples is the key to learn robust video representations.
  arXiv  Detail & Related papers  (2021-06-04T08:44:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.