Generative Spatiotemporal Data Augmentation

• URL: http://arxiv.org/abs/2512.12508v1
• Date: Sun, 14 Dec 2025 01:18:48 GMT
• Title: Generative Spatiotemporal Data Augmentation
• Authors: Jinfan Zhou, Lixin Luo, Sungmin Eum, Heesung Kwon, Jeong Joon Park,
• Abstract summary: We use video foundation models to generate realistic 3D spatial and temporal variations from an image dataset.<n> Incorporating synthesized video clips as supplemental data yields consistent performance gains in low-data settings.
• Score: 12.849046721804797
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We explore spatiotemporal data augmentation using video foundation models to diversify both camera viewpoints and scene dynamics. Unlike existing approaches based on simple geometric transforms or appearance perturbations, our method leverages off-the-shelf video diffusion models to generate realistic 3D spatial and temporal variations from a given image dataset. Incorporating these synthesized video clips as supplemental training data yields consistent performance gains in low-data settings, such as UAV-captured imagery where annotations are scarce. Beyond empirical improvements, we provide practical guidelines for (i) choosing an appropriate spatiotemporal generative setup, (ii) transferring annotations to synthetic frames, and (iii) addressing disocclusion - regions newly revealed and unlabeled in generated views. Experiments on COCO subsets and UAV-captured datasets show that, when applied judiciously, spatiotemporal augmentation broadens the data distribution along axes underrepresented by traditional and prior generative methods, offering an effective lever for improving model performance in data-scarce regimes.

Related papers

• BetterScene: 3D Scene Synthesis with Representation-Aligned Generative Model [3.7515646463759698]
  We present BetterScene, an approach to enhance novel view synthesis (NVS) quality for diverse real-world scenes using extremely sparse, unconstrained photos.<n>BetterScene leverages the production-ready Stable Video Diffusion (SVD) model pretrained on billions of frames as a strong backbone.<n>We evaluate on the challenging DL3DV-10K dataset and demonstrate superior performance compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-02-26T03:58:42Z)
• Visual Autoregressive Modelling for Monocular Depth Estimation [69.01449528371916]
  We propose a monocular depth estimation method based on visual autoregressive ( VAR) priors.<n>Our method adapts a large-scale text-to-image VAR model and introduces a scale-wise conditional upsampling mechanism.<n>We report state-of-the-art performance in indoor benchmarks under constrained training conditions, and strong performance when applied to outdoor datasets.
  arXiv  Detail & Related papers  (2025-12-27T17:08:03Z)
• Diff-3DCap: Shape Captioning with Diffusion Models [27.69808457236316]
  We introduce Diff-3DCap, which employs a sequence of projected views to represent a 3D object and a continuous diffusion model to facilitate the captioning process.<n>Results of our experiments indicate that Diff-3DCap can achieve performance comparable to that of the current state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-09-28T07:59:22Z)
• SPIRAL: Semantic-Aware Progressive LiDAR Scene Generation and Understanding [25.20103686487509]
  Spiral is a novel range-view LiDAR diffusion model that simultaneously generates depth, reflectance images, and semantic maps.<n> Experiments on the Semantic KITTI and nuScenes datasets demonstrate that Spiral achieves state-of-the-art performance with the smallest parameter size.
  arXiv  Detail & Related papers  (2025-05-28T17:55:35Z)
• Intern-GS: Vision Model Guided Sparse-View 3D Gaussian Splatting [95.61137026932062]
  Intern-GS is a novel approach to enhance the process of sparse-view Gaussian splatting.<n>We show that Intern-GS achieves state-of-the-art rendering quality across diverse datasets.
  arXiv  Detail & Related papers  (2025-05-27T05:17:49Z)
• DiffusionSfM: Predicting Structure and Motion via Ray Origin and Endpoint Diffusion [53.70278210626701]
  We propose a data-driven multi-view reasoning approach that directly infers 3D scene geometry and camera poses from multi-view images.<n>Our framework, DiffusionSfM, parameterizes scene geometry and cameras as pixel-wise ray origins and endpoints in a global frame.<n>We empirically validate DiffusionSfM on both synthetic and real datasets, demonstrating that it outperforms classical and learning-based approaches.
  arXiv  Detail & Related papers  (2025-05-08T17:59:47Z)
• AccVideo: Accelerating Video Diffusion Model with Synthetic Dataset [55.82208863521353]
  We propose AccVideo to reduce the inference steps for accelerating video diffusion models with synthetic dataset.<n>Our model achieves 8.5x improvements in generation speed compared to the teacher model.<n>Compared to previous accelerating methods, our approach is capable of generating videos with higher quality and resolution.
  arXiv  Detail & Related papers  (2025-03-25T08:52:07Z)
• 4Diffusion: Multi-view Video Diffusion Model for 4D Generation [55.82208863521353]
  Current 4D generation methods have achieved noteworthy efficacy with the aid of advanced diffusion generative models. We propose a novel 4D generation pipeline, namely 4Diffusion, aimed at generating spatial-temporally consistent 4D content from a monocular video.
  arXiv  Detail & Related papers  (2024-05-31T08:18:39Z)
• Fast LiDAR Upsampling using Conditional Diffusion Models [1.3709133749179265]
  Existing approaches have shown the possibilities for using diffusion models to generate refined LiDAR data with high fidelity. We introduce a novel approach based on conditional diffusion models for fast and high-quality sparse-to-dense upsampling of 3D scene point clouds. Our method employs denoising diffusion probabilistic models trained with conditional inpainting masks, which have been shown to give high performance on image completion tasks.
  arXiv  Detail & Related papers  (2024-05-08T08:38:28Z)
• Steerable Conditional Diffusion for Out-of-Distribution Adaptation in Medical Image Reconstruction [75.91471250967703]
  We introduce a novel sampling framework called Steerable Conditional Diffusion.<n>This framework adapts the diffusion model, concurrently with image reconstruction, based solely on the information provided by the available measurement.<n>We achieve substantial enhancements in out-of-distribution performance across diverse imaging modalities.
  arXiv  Detail & Related papers  (2023-08-28T08:47:06Z)
• Deceptive-NeRF/3DGS: Diffusion-Generated Pseudo-Observations for High-Quality Sparse-View Reconstruction [60.52716381465063]
  We introduce Deceptive-NeRF/3DGS to enhance sparse-view reconstruction with only a limited set of input images. Specifically, we propose a deceptive diffusion model turning noisy images rendered from few-view reconstructions into high-quality pseudo-observations. Our system progressively incorporates diffusion-generated pseudo-observations into the training image sets, ultimately densifying the sparse input observations by 5 to 10 times.
  arXiv  Detail & Related papers  (2023-05-24T14:00:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.