Hyper Diffusion Avatars: Dynamic Human Avatar Generation using Network Weight Space Diffusion

• URL: http://arxiv.org/abs/2509.04145v2
• Date: Mon, 08 Sep 2025 08:40:58 GMT
• Title: Hyper Diffusion Avatars: Dynamic Human Avatar Generation using Network Weight Space Diffusion
• Authors: Dongliang Cao, Guoxing Sun, Marc Habermann, Florian Bernard,
• Abstract summary: We propose a novel approach that unites the strengths of person-specific rendering and diffusion-based generative modeling.<n>Our method follows a two-stage pipeline: first, we optimize a set of person-specific UNets, with each network representing a dynamic human avatar.<n>During inference, our method generates network weights for real-time, controllable rendering of dynamic human avatars.
• Score: 45.88321772203678
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Creating human avatars is a highly desirable yet challenging task. Recent advancements in radiance field rendering have achieved unprecedented photorealism and real-time performance for personalized dynamic human avatars. However, these approaches are typically limited to person-specific rendering models trained on multi-view video data for a single individual, limiting their ability to generalize across different identities. On the other hand, generative approaches leveraging prior knowledge from pre-trained 2D diffusion models can produce cartoonish, static human avatars, which are animated through simple skeleton-based articulation. Therefore, the avatars generated by these methods suffer from lower rendering quality compared to person-specific rendering methods and fail to capture pose-dependent deformations such as cloth wrinkles. In this paper, we propose a novel approach that unites the strengths of person-specific rendering and diffusion-based generative modeling to enable dynamic human avatar generation with both high photorealism and realistic pose-dependent deformations. Our method follows a two-stage pipeline: first, we optimize a set of person-specific UNets, with each network representing a dynamic human avatar that captures intricate pose-dependent deformations. In the second stage, we train a hyper diffusion model over the optimized network weights. During inference, our method generates network weights for real-time, controllable rendering of dynamic human avatars. Using a large-scale, cross-identity, multi-view video dataset, we demonstrate that our approach outperforms state-of-the-art human avatar generation methods.

Related papers

• PERSONA: Personalized Whole-Body 3D Avatar with Pose-Driven Deformations from a Single Image [17.76649311703262]
  Two major approaches exist for creating animatable human avatars.<n>A 3D-based approach achieves personalization through a disentangled identity representation.<n>A diffusion-based approach learns pose-driven deformations from large-scale in-the-wild videos but struggles with identity preservation.<n>We present PERSONA, a framework that combines the strengths of both approaches to obtain a personalized 3D human avatar.
  arXiv  Detail & Related papers  (2025-08-13T17:40:48Z)
• EVA: Expressive Virtual Avatars from Multi-view Videos [51.33851869426057]
  We introduce Expressive Virtual Avatars (EVA), an actor-specific, fully controllable, and expressive human avatar framework.<n>EVA achieves high-fidelity, lifelike renderings in real time while enabling independent control of facial expressions, body movements, and hand gestures.<n>This work represents a significant advancement towards fully drivable digital human models.
  arXiv  Detail & Related papers  (2025-05-21T11:22:52Z)
• Multimodal Generation of Animatable 3D Human Models with AvatarForge [67.31920821192323]
  AvatarForge is a framework for generating animatable 3D human avatars from text or image inputs using AI-driven procedural generation.<n>Our evaluations show that AvatarForge outperforms state-of-the-art methods in both text- and image-to-avatar generation.
  arXiv  Detail & Related papers  (2025-03-11T08:29:18Z)
• WonderHuman: Hallucinating Unseen Parts in Dynamic 3D Human Reconstruction [63.666166249633385]
  We present WonderHuman to reconstruct dynamic human avatars from a monocular video for high-fidelity novel view synthesis.<n>Our method achieves SOTA performance in producing photorealistic renderings from the given monocular video.
  arXiv  Detail & Related papers  (2025-02-03T04:43:41Z)
• GaussianAvatar: Towards Realistic Human Avatar Modeling from a Single Video via Animatable 3D Gaussians [51.46168990249278]
  We present an efficient approach to creating realistic human avatars with dynamic 3D appearances from a single video. GustafAvatar is validated on both the public dataset and our collected dataset.
  arXiv  Detail & Related papers  (2023-12-04T18:55:45Z)
• AvatarGen: a 3D Generative Model for Animatable Human Avatars [108.11137221845352]
  AvatarGen is the first method that enables not only non-rigid human generation with diverse appearance but also full control over poses and viewpoints. To model non-rigid dynamics, it introduces a deformation network to learn pose-dependent deformations in the canonical space. Our method can generate animatable human avatars with high-quality appearance and geometry modeling, significantly outperforming previous 3D GANs.
  arXiv  Detail & Related papers  (2022-08-01T01:27:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.