Exploring the Role of Synthetic Data Augmentation in Controllable Human-Centric Video Generation

• URL: http://arxiv.org/abs/2604.21291v1
• Date: Thu, 23 Apr 2026 05:10:15 GMT
• Title: Exploring the Role of Synthetic Data Augmentation in Controllable Human-Centric Video Generation
• Abstract summary: Controllable human video generation aims to produce realistic videos of humans with explicitly guided motions and appearances.<n>Synthetic data provides a scalable and controllable alternative.<n>Our study offers the first comprehensive exploration of synthetic data's role in human-centric video synthesis.
• Score: 9.875136212706165
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Controllable human video generation aims to produce realistic videos of humans with explicitly guided motions and appearances,serving as a foundation for digital humans, animation, and embodied AI.However, the scarcity of largescale, diverse, and privacy safe human video datasets poses a major bottleneck, especially for rare identities and complex actions.Synthetic data provides a scalable and controllable alternative,yet its actual contribution to generative modeling remains underexplored due to the persistent Sim2Real gap.In this work,we systematically investigate the impact of synthetic data on controllable human video generation. We propose a diffusion-based framework that enables fine-grained control over appearance and motion while providing a unfied testbed to analyze how synthetic data interacts with real world data during training. Through extensive experiments, we reveal the complementary roles of synthetic and real data and demonstrate possible methods for efficiently selecting synthetic samples to enhance motion realism,temporal consistency,and identity preservation.Our study offers the first comprehensive exploration of synthetic data's role in human-centric video synthesis and provides practical insights for building data-efficient and generalizable generative models.

Related papers

• RoboDream: Compositional World Models for Scalable Robot Data Synthesis [31.864551794584667]
  Scaling robot learning requires large-scale, diverse demonstrations, yet real-world data collection via teleoperation remains prohibitively expensive and time-consuming.<n>We present a generalizable embodiment-centric world model that achieves scalable data generation by synthesizing demonstrations with novel objects.<n>We demonstrate with real-world experiments that our generated data consistently improves downstream policy performance and significantly reduces real-world data requirements across diverse manipulation tasks.
  arXiv  Detail & Related papers  (2026-06-01T17:59:38Z)
• Prompt-to-Gesture: Measuring the Capabilities of Image-to-Video Deictic Gesture Generation [8.9657942429828]
  Recent advancements in image-to-video foundation models have enabled the generation of photorealistic, semantically rich videos guided by natural language.<n>These capabilities open up new possibilities for creating effort-free synthetic data.<n>In this paper, we introduce and analyze prompt-based video generation to construct a realistic deictic gestures dataset.
  arXiv  Detail & Related papers  (2026-04-16T12:52:12Z)
• AnchorDream: Repurposing Video Diffusion for Embodiment-Aware Robot Data Synthesis [33.90053396451562]
  We introduce AnchorDream, an embodiment-aware world model that repurposes pretrained video diffusion models for robot data synthesis.<n>Our method scales them into large, diverse, high-quality datasets without requiring explicit environment modeling.<n>Experiments show that the generated data leads to consistent improvements in downstream policy learning, with relative gains of 36.4% in simulator benchmarks and nearly double performance in real-world studies.
  arXiv  Detail & Related papers  (2025-12-12T18:59:45Z)
• Understanding the Influence of Synthetic Data for Text Embedders [52.04771455432998]
  We first reproduce and publicly release the synthetic data proposed by Wang et al.<n>We critically examine where exactly synthetic data improves model generalization.<n>Our findings highlight the limitations of current synthetic data approaches for building general-purpose embedders.
  arXiv  Detail & Related papers  (2025-09-07T19:28:52Z)
• SynFER: Towards Boosting Facial Expression Recognition with Synthetic Data [78.70620682374624]
  We introduce SynFER, a novel framework for synthesizing facial expression image data based on high-level textual descriptions.<n>To ensure the quality and reliability of the synthetic data, we propose a semantic guidance technique and a pseudo-label generator.<n>Results validate the efficacy of our approach and the synthetic data.
  arXiv  Detail & Related papers  (2024-10-13T14:58:21Z)
• Best Practices and Lessons Learned on Synthetic Data [83.63271573197026]
  The success of AI models relies on the availability of large, diverse, and high-quality datasets. Synthetic data has emerged as a promising solution by generating artificial data that mimics real-world patterns.
  arXiv  Detail & Related papers  (2024-04-11T06:34:17Z)
• Scaling Up Dynamic Human-Scene Interaction Modeling [58.032368564071895]
  TRUMANS is the most comprehensive motion-captured HSI dataset currently available. It intricately captures whole-body human motions and part-level object dynamics. We devise a diffusion-based autoregressive model that efficiently generates HSI sequences of any length.
  arXiv  Detail & Related papers  (2024-03-13T15:45:04Z)
• Learning Human Action Recognition Representations Without Real Humans [66.61527869763819]
  We present a benchmark that leverages real-world videos with humans removed and synthetic data containing virtual humans to pre-train a model. We then evaluate the transferability of the representation learned on this data to a diverse set of downstream action recognition benchmarks. Our approach outperforms previous baselines by up to 5%.
  arXiv  Detail & Related papers  (2023-11-10T18:38:14Z)
• Training Robust Deep Physiological Measurement Models with Synthetic Video-based Data [11.31971398273479]
  We propose measures to add real-world noise to synthetic physiological signals and corresponding facial videos. Our results show that we were able to reduce the average MAE from 6.9 to 2.0.
  arXiv  Detail & Related papers  (2023-11-09T13:55:45Z)
• Synthetic-to-Real Domain Adaptation for Action Recognition: A Dataset and Baseline Performances [76.34037366117234]
  We introduce a new dataset called Robot Control Gestures (RoCoG-v2) The dataset is composed of both real and synthetic videos from seven gesture classes. We present results using state-of-the-art action recognition and domain adaptation algorithms.
  arXiv  Detail & Related papers  (2023-03-17T23:23:55Z)
• Procedural Humans for Computer Vision [1.9550079119934403]
  We build a parametric model of the face and body, including articulated hands, to generate realistic images of humans based on this body model. We show that this can be extended to include the full body by building on the pipeline of Wood et al. to generate synthetic images of humans in their entirety.
  arXiv  Detail & Related papers  (2023-01-03T15:44:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.