DiffAge3D: Diffusion-based 3D-aware Face Aging

• URL: http://arxiv.org/abs/2408.15922v1
• Date: Wed, 28 Aug 2024 16:36:09 GMT
• Title: DiffAge3D: Diffusion-based 3D-aware Face Aging
• Authors: Junaid Wahid, Fangneng Zhan, Pramod Rao, Christian Theobalt,
• Abstract summary: We propose DiffAge3D, the first 3D-aware aging framework that performs faithful aging and identity preservation in a 3D setting. Our framework includes a robust 3D-aware aging dataset generation pipeline by utilizing a pre-trained 3D GAN. We demonstrate that DiffAge3D outperforms existing methods, particularly in multiview-consistent aging and fine details preservation.
• Score: 61.3027596093854
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Face aging is the process of converting an individual's appearance to a younger or older version of themselves. Existing face aging techniques have been limited to 2D settings, which often weaken their applications as there is a growing demand for 3D face modeling. Moreover, existing aging methods struggle to perform faithful aging, maintain identity, and retain the fine details of the input images. Given these limitations and the need for a 3D-aware aging method, we propose DiffAge3D, the first 3D-aware aging framework that not only performs faithful aging and identity preservation but also operates in a 3D setting. Our aging framework allows to model the aging and camera pose separately by only taking a single image with a target age. Our framework includes a robust 3D-aware aging dataset generation pipeline by utilizing a pre-trained 3D GAN and the rich text embedding capabilities within CLIP model. Notably, we do not employ any inversion bottleneck in dataset generation. Instead, we randomly generate training samples from the latent space of 3D GAN, allowing us to manipulate the rich latent space of GAN to generate ages even with large gaps. With the generated dataset, we train a viewpoint-aware diffusion-based aging model to control the camera pose and facial age. Through quantitative and qualitative evaluations, we demonstrate that DiffAge3D outperforms existing methods, particularly in multiview-consistent aging and fine details preservation.

Related papers

• ID-to-3D: Expressive ID-guided 3D Heads via Score Distillation Sampling [96.87575334960258]
  ID-to-3D is a method to generate identity- and text-guided 3D human heads with disentangled expressions. Results achieve an unprecedented level of identity-consistent and high-quality texture and geometry generation.
  arXiv  Detail & Related papers  (2024-05-26T13:36:45Z)
• 3D-SceneDreamer: Text-Driven 3D-Consistent Scene Generation [51.64796781728106]
  We propose a generative refinement network to synthesize new contents with higher quality by exploiting the natural image prior to 2D diffusion model and the global 3D information of the current scene. Our approach supports wide variety of scene generation and arbitrary camera trajectories with improved visual quality and 3D consistency.
  arXiv  Detail & Related papers  (2024-03-14T14:31:22Z)
• Face Aging via Diffusion-based Editing [5.318584973533008]
  We propose FADING, a novel approach to address Face Aging via DIffusion-based editiNG. We go beyond existing methods by leveraging the rich prior of large-scale language-image diffusion models. Our method outperforms existing approaches with respect to aging accuracy, attribute preservation, and aging quality.
  arXiv  Detail & Related papers  (2023-09-20T13:47:10Z)
• A Generative Framework for Self-Supervised Facial Representation Learning [18.094262972295702]
  Self-supervised representation learning has gained increasing attention for strong generalization ability without relying on paired datasets. Self-supervised facial representation learning remains unsolved due to the coupling of facial identities, expressions, and external factors like pose and light. We propose LatentFace, a novel generative framework for self-supervised facial representations.
  arXiv  Detail & Related papers  (2023-09-15T09:34:05Z)
• CGOF++: Controllable 3D Face Synthesis with Conditional Generative Occupancy Fields [52.14985242487535]
  We propose a new conditional 3D face synthesis framework, which enables 3D controllability over generated face images. At its core is a conditional Generative Occupancy Field (cGOF++) that effectively enforces the shape of the generated face to conform to a given 3D Morphable Model (3DMM) mesh. Experiments validate the effectiveness of the proposed method and show more precise 3D controllability than state-of-the-art 2D-based controllable face synthesis methods.
  arXiv  Detail & Related papers  (2022-11-23T19:02:50Z)
• Controllable 3D Generative Adversarial Face Model via Disentangling Shape and Appearance [63.13801759915835]
  3D face modeling has been an active area of research in computer vision and computer graphics. This paper proposes a new 3D face generative model that can decouple identity and expression.
  arXiv  Detail & Related papers  (2022-08-30T13:40:48Z)
• OSTeC: One-Shot Texture Completion [86.23018402732748]
  We propose an unsupervised approach for one-shot 3D facial texture completion. The proposed approach rotates an input image in 3D and fill-in the unseen regions by reconstructing the rotated image in a 2D face generator. We frontalize the target image by projecting the completed texture into the generator.
  arXiv  Detail & Related papers  (2020-12-30T23:53:26Z)
• Rotate-and-Render: Unsupervised Photorealistic Face Rotation from Single-View Images [47.18219551855583]
  We propose a novel unsupervised framework that can synthesize photo-realistic rotated faces. Our key insight is that rotating faces in the 3D space back and forth, and re-rendering them to the 2D plane can serve as a strong self-supervision. Our approach has superior synthesis quality as well as identity preservation over the state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-03-18T09:54:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.