One-Shot Novel View and Pose Human Image Synthesis via 3D Prior Guided Diffusion Model

• URL: http://arxiv.org/abs/2606.19718v1
• Date: Thu, 18 Jun 2026 02:34:24 GMT
• Title: One-Shot Novel View and Pose Human Image Synthesis via 3D Prior Guided Diffusion Model
• Abstract summary: We propose a novel approach for novel view and pose synthesis from a singe human image via conditional denoising diffusion model.<n>Our diffusion model divides the novel view and pose synthesis problem into a sequence of conditional denoising steps.<n>By transferring the reference human into the target human with a series of diffusion steps, our diffusion model enables high-quality synthesis.
• Score: 21.980221869687508
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper addresses the challenge of one-shot novel view and pose human image synthesis. The existing methods transfer the reference human image to a target pose using a set of 2D pose keypoints or synthesize human images based on generalizable human NeRF which uses human model priors to extract point-wise features. However, pose transfer based methods can not handle complex human pose using ambiguous 2D pose as the condition, while generalizable human NeRFs may be inaccurate to recover occluded/invisiable human parts without extracted reliable features. To solve these problems, we propose a novel approach for novel view and pose synthesis from a singe human image via conditional denoising diffusion model. Our diffusion model divides the novel view and pose synthesis problem into a sequence of conditional denoising steps. Specifically, to generate humans with complex and arbitrary poses, we introduce 3D human priors, i.e., 3D normal map and color prompt, as geometry and color conditions into the generation process. By transferring the reference human into the target human with a series of diffusion steps, our diffusion model enables high-quality synthesis including the occluded/invisible parts. Further, we propose a self-reconstruction based customized refinement to enhance fine details when tested on novel persons.Experimental results on different public datasets demonstrate that our approach significantly outperforms previous methods and also shows better generalization ability across datasets. The code will be made publicly available at https://github.com/Yankeegsj/3DPGDM.

Related papers

• DPoser-X: Diffusion Model as Robust 3D Whole-body Human Pose Prior [82.9526308672547]
  We present DPoser-X, a diffusion-based prior model for 3D whole-body human poses.<n>Our approach unifies various pose-centric tasks as inverse problems, solving them through variational diffusion sampling.<n>Our model consistently outperforms state-of-the-art alternatives, establishing a new benchmark for whole-body human pose prior modeling.
  arXiv  Detail & Related papers  (2025-08-01T12:56:39Z)
• HumanGif: Single-View Human Diffusion with Generative Prior [25.516544735593087]
  We propose HumanGif, a single-view human diffusion model with generative priors.<n>Specifically, we formulate the single-view-based 3D human novel view and pose synthesis as a single-view-conditioned human diffusion process.<n>We show that HumanGif achieves the best perceptual performance, with better generalizability for novel view and pose synthesis.
  arXiv  Detail & Related papers  (2025-02-17T17:55:27Z)
• GAS: Generative Avatar Synthesis from a Single Image [54.95198111659466]
  We present a framework for synthesizing view-consistent and temporally coherent avatars from a single image.<n>Our approach combines the reconstruction power of regression-based 3D human reconstruction with the generative capabilities of a diffusion model.
  arXiv  Detail & Related papers  (2025-02-10T19:00:39Z)
• PSHuman: Photorealistic Single-image 3D Human Reconstruction using Cross-Scale Multiview Diffusion and Explicit Remeshing [47.191113407993015]
  PSHuman is a novel framework that explicitly reconstructs human meshes utilizing priors from the multiview diffusion model.<n>It is found that directly applying multiview diffusion on single-view human images leads to severe geometric distortions.<n>To enhance cross-view body shape consistency of varied human poses, we condition the generative model on parametric models like SMPL-X.
  arXiv  Detail & Related papers  (2024-09-16T10:13:06Z)
• InceptionHuman: Controllable Prompt-to-NeRF for Photorealistic 3D Human Generation [61.62346472443454]
  InceptionHuman is a prompt-to-NeRF framework that allows easy control via a combination of prompts in different modalities to generate photorealistic 3D humans. InceptionHuman achieves consistent 3D human generation within a progressively refined NeRF space.
  arXiv  Detail & Related papers  (2023-11-27T15:49:41Z)
• Novel View Synthesis of Humans using Differentiable Rendering [50.57718384229912]
  We present a new approach for synthesizing novel views of people in new poses. Our synthesis makes use of diffuse Gaussian primitives that represent the underlying skeletal structure of a human. Rendering these primitives gives results in a high-dimensional latent image, which is then transformed into an RGB image by a decoder network.
  arXiv  Detail & Related papers  (2023-03-28T10:48:33Z)
• Neural Novel Actor: Learning a Generalized Animatable Neural Representation for Human Actors [98.24047528960406]
  We propose a new method for learning a generalized animatable neural representation from a sparse set of multi-view imagery of multiple persons. The learned representation can be used to synthesize novel view images of an arbitrary person from a sparse set of cameras, and further animate them with the user's pose control.
  arXiv  Detail & Related papers  (2022-08-25T07:36:46Z)
• NeuralReshaper: Single-image Human-body Retouching with Deep Neural Networks [50.40798258968408]
  We present NeuralReshaper, a novel method for semantic reshaping of human bodies in single images using deep generative networks. Our approach follows a fit-then-reshape pipeline, which first fits a parametric 3D human model to a source human image. To deal with the lack-of-data problem that no paired data exist, we introduce a novel self-supervised strategy to train our network.
  arXiv  Detail & Related papers  (2022-03-20T09:02:13Z)
• Liquid Warping GAN with Attention: A Unified Framework for Human Image Synthesis [58.05389586712485]
  We tackle human image synthesis, including human motion imitation, appearance transfer, and novel view synthesis. In this paper, we propose a 3D body mesh recovery module to disentangle the pose and shape. We also build a new dataset, namely iPER dataset, for the evaluation of human motion imitation, appearance transfer, and novel view synthesis.
  arXiv  Detail & Related papers  (2020-11-18T02:57:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.