PoseDreamer: Scalable and Photorealistic Human Data Generation Pipeline with Diffusion Models

• URL: http://arxiv.org/abs/2603.28763v1
• Date: Mon, 30 Mar 2026 17:59:18 GMT
• Title: PoseDreamer: Scalable and Photorealistic Human Data Generation Pipeline with Diffusion Models
• Abstract summary: PoseDreamer is a novel pipeline that generates synthetic datasets with 3D mesh annotations.<n>We generate more than 500,000 high-quality synthetic samples, achieving a 76% improvement in image-quality metrics.
• Score: 41.23359540842301
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Acquiring labeled datasets for 3D human mesh estimation is challenging due to depth ambiguities and the inherent difficulty of annotating 3D geometry from monocular images. Existing datasets are either real, with manually annotated 3D geometry and limited scale, or synthetic, rendered from 3D engines that provide precise labels but suffer from limited photorealism, low diversity, and high production costs. In this work, we explore a third path: generated data. We introduce PoseDreamer, a novel pipeline that leverages diffusion models to generate large-scale synthetic datasets with 3D mesh annotations. Our approach combines controllable image generation with Direct Preference Optimization for control alignment, curriculum-based hard sample mining, and multi-stage quality filtering. Together, these components naturally maintain correspondence between 3D labels and generated images, while prioritizing challenging samples to maximize dataset utility. Using PoseDreamer, we generate more than 500,000 high-quality synthetic samples, achieving a 76% improvement in image-quality metrics compared to rendering-based datasets. Models trained on PoseDreamer achieve performance comparable to or superior to those trained on real-world and traditional synthetic datasets. In addition, combining PoseDreamer with synthetic datasets results in better performance than combining real-world and synthetic datasets, demonstrating the complementary nature of our dataset. We will release the full dataset and generation code.

Related papers

• BlendFusion -- Scalable Synthetic Data Generation for Diffusion Model Training [0.07419401583272026]
  We propose BlendFusion, a framework for synthetic data generation from 3D scenes using path tracing.<n>Our pipeline incorporates an object-centric camera placement strategy, robust filtering mechanisms, and automatic captioning.<n>We empirically analyze the quality of FineBLEND, an image-caption dataset constructed from a diverse set of 3D scenes.
  arXiv  Detail & Related papers  (2026-04-10T06:36:38Z)
• A Framework for Low-Effort Training Data Generation for Urban Semantic Segmentation [15.541453405140485]
  Synthetic datasets are widely used for training urban scene recognition models, but even highly realistic renderings show a noticeable gap to real imagery.<n>We present a new framework that adapts an off-the-shelf diffusion model to a target domain using only imperfect pseudo-labels.<n>It generates high-fidelity, target-aligned images from semantic maps of any synthetic dataset, including low-effort sources created in hours rather than months.
  arXiv  Detail & Related papers  (2025-10-13T16:12:29Z)
• Drive-1-to-3: Enriching Diffusion Priors for Novel View Synthesis of Real Vehicles [81.29018359825872]
  This paper consolidates a set of good practices to finetune large pretrained models for a real-world task.<n>Specifically, we develop several strategies to account for discrepancies between the synthetic data and real driving data.<n>Our insights lead to effective finetuning that results in a $68.8%$ reduction in FID for novel view synthesis over prior arts.
  arXiv  Detail & Related papers  (2024-12-19T03:39:13Z)
• Diffusion Models are Efficient Data Generators for Human Mesh Recovery [55.37787289869703]
  We show that synthetic data created by generative models is complementary to CG-rendered data.<n>We propose an effective data generation pipeline based on recent diffusion models, termed HumanWild.<n>Our work could pave the way for scaling up 3D human recovery to in-the-wild scenes.
  arXiv  Detail & Related papers  (2024-03-17T06:31:16Z)
• UniG3D: A Unified 3D Object Generation Dataset [75.49544172927749]
  UniG3D is a unified 3D object generation dataset constructed by employing a universal data transformation pipeline on ShapeNet datasets. This pipeline converts each raw 3D model into comprehensive multi-modal data representation. The selection of data sources for our dataset is based on their scale and quality.
  arXiv  Detail & Related papers  (2023-06-19T07:03:45Z)
• RTMV: A Ray-Traced Multi-View Synthetic Dataset for Novel View Synthesis [104.53930611219654]
  We present a large-scale synthetic dataset for novel view synthesis consisting of 300k images rendered from nearly 2000 complex scenes. The dataset is orders of magnitude larger than existing synthetic datasets for novel view synthesis. Using 4 distinct sources of high-quality 3D meshes, the scenes of our dataset exhibit challenging variations in camera views, lighting, shape, materials, and textures.
  arXiv  Detail & Related papers  (2022-05-14T13:15:32Z)
• STPLS3D: A Large-Scale Synthetic and Real Aerial Photogrammetry 3D Point Cloud Dataset [6.812704277866377]
  We introduce a synthetic aerial photogrammetry point clouds generation pipeline. Unlike generating synthetic data in virtual games, the proposed pipeline simulates the reconstruction process of the real environment. We present a richly-annotated synthetic 3D aerial photogrammetry point cloud dataset.
  arXiv  Detail & Related papers  (2022-03-17T03:50:40Z)
• UltraPose: Synthesizing Dense Pose with 1 Billion Points by Human-body Decoupling 3D Model [58.70130563417079]
  We introduce a new 3D human-body model with a series of decoupled parameters that could freely control the generation of the body. Compared to the existing manually annotated DensePose-COCO dataset, the synthetic UltraPose has ultra dense image-to-surface correspondences without annotation cost and error.
  arXiv  Detail & Related papers  (2021-10-28T16:24:55Z)
• Synthetic Data and Hierarchical Object Detection in Overhead Imagery [0.0]
  We develop novel synthetic data generation and augmentation techniques for enhancing low/zero-sample learning in satellite imagery. To test the effectiveness of synthetic imagery, we employ it in the training of detection models and our two stage model, and evaluate the resulting models on real satellite images.
  arXiv  Detail & Related papers  (2021-01-29T22:52:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.