Data Warmup: Complexity-Aware Curricula for Efficient Diffusion Training

• URL: http://arxiv.org/abs/2604.07397v1
• Date: Wed, 08 Apr 2026 08:55:49 GMT
• Title: Data Warmup: Complexity-Aware Curricula for Efficient Diffusion Training
• Abstract summary: We propose Data Warmup, a curriculum strategy that schedules training images from simple to complex without modifying the model or loss.<n>On ImageNet 256x256 with SiT backbones, Data Warmup improves IS by up to 6.11 and FID by up to 3.41, reaching baseline quality tens of thousands of iterations earlier.
• Score: 12.295531883770622
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: A key inefficiency in diffusion training occurs when a randomly initialized network, lacking visual priors, encounters gradients from the full complexity spectrum--most of which it lacks the capacity to resolve. We propose Data Warmup, a curriculum strategy that schedules training images from simple to complex without modifying the model or loss. Each image is scored offline by a semantic-aware complexity metric combining foreground dominance (how much of the image salient objects occupy) and foreground typicality (how closely the salient content matches learned visual prototypes). A temperature-controlled sampler then prioritizes low-complexity images early and anneals toward uniform sampling. On ImageNet 256x256 with SiT backbones (S/2 to XL/2), Data Warmup improves IS by up to 6.11 and FID by up to 3.41, reaching baseline quality tens of thousands of iterations earlier. Reversing the curriculum (exposing hard images first) degrades performance below the uniform baseline, confirming that the simple-to-complex ordering itself drives the gains. The method combines with orthogonal accelerators such as REPA and requires only ~10 minutes of one-time preprocessing with zero per-iteration overhead.

Related papers

• Representation Entanglement for Generation: Training Diffusion Transformers Is Much Easier Than You Think [63.25744258438214]
  REPA and its variants effectively mitigate training challenges in diffusion models by incorporating external visual representations from pretrained models.<n>We argue that the external alignment, which is absent during the entire denoising inference process, falls short of fully harnessing the potential of discriminative representations.<n>We propose Representation Entanglement for Generation (REG), which entangles low-level image latents with a single high-level class token from pretrained foundation models for denoising.
  arXiv  Detail & Related papers  (2025-07-02T08:29:18Z)
• REPA Works Until It Doesn't: Early-Stopped, Holistic Alignment Supercharges Diffusion Training [58.33728862521732]
  Diffusion Transformers (DiTs) deliver state-of-the-art image quality, yet their training remains notoriously slow.<n>A recent remedy -- representation alignment (REPA) that matches DiT hidden features to those of a non-generative teacher (e.g. DINO) -- dramatically accelerates the early epochs but plateaus or even degrades performance later.<n>We trace this failure to a capacity mismatch: once the generative student begins modelling the joint data distribution, the teacher's lower-dimensional embeddings and attention patterns become a straitjacket rather than a guide.<n>We introduce HASTE
  arXiv  Detail & Related papers  (2025-05-22T15:34:33Z)
• Towards Better Alignment: Training Diffusion Models with Reinforcement Learning Against Sparse Rewards [52.90573877727541]
  reinforcement learning (RL) has been considered for diffusion model fine-tuning.<n>RL's effectiveness is limited by the challenge of sparse reward.<n>$textB2text-DiffuRL$ is compatible with existing optimization algorithms.
  arXiv  Detail & Related papers  (2025-03-14T09:45:19Z)
• Data-Efficient Generation for Dataset Distillation [12.106527496044473]
  We train a conditional latent diffusion model capable of generating realistic synthetic images with labels. We demonstrate that models can be effectively trained using only a small set of synthetic images and evaluated on a large real test set.
  arXiv  Detail & Related papers  (2024-09-05T22:31:53Z)
• One-Shot Image Restoration [0.0]
  Experimental results demonstrate the applicability, robustness and computational efficiency of the proposed approach for supervised image deblurring and super-resolution. Our results showcase significant improvement of learning models' sample efficiency, generalization and time complexity.
  arXiv  Detail & Related papers  (2024-04-26T14:03:23Z)
• Simultaneous Image-to-Zero and Zero-to-Noise: Diffusion Models with Analytical Image Attenuation [53.04220377034574]
  We propose incorporating an analytical image attenuation process into the forward diffusion process for high-quality (un)conditioned image generation.<n>Our method represents the forward image-to-noise mapping as simultaneous textitimage-to-zero mapping and textitzero-to-noise mapping.<n>We have conducted experiments on unconditioned image generation, textite.g., CIFAR-10 and CelebA-HQ-256, and image-conditioned downstream tasks such as super-resolution, saliency detection, edge detection, and image inpainting.
  arXiv  Detail & Related papers  (2023-06-23T18:08:00Z)
• FastMIM: Expediting Masked Image Modeling Pre-training for Vision [65.47756720190155]
  FastMIM is a framework for pre-training vision backbones with low-resolution input images. It reconstructs Histograms of Oriented Gradients (HOG) feature instead of original RGB values of the input images. It can achieve 83.8%/84.1% top-1 accuracy on ImageNet-1K with ViT-B/Swin-B as backbones.
  arXiv  Detail & Related papers  (2022-12-13T14:09:32Z)
• Noise Self-Regression: A New Learning Paradigm to Enhance Low-Light Images Without Task-Related Data [86.68013790656762]
  We propose Noise SElf-Regression (NoiSER) without access to any task-related data.<n>NoiSER is highly competitive in enhancement quality, yet with a much smaller model size, and much lower training and inference cost.
  arXiv  Detail & Related papers  (2022-11-09T06:18:18Z)
• Deep Amended Gradient Descent for Efficient Spectral Reconstruction from Single RGB Images [42.26124628784883]
  We propose a compact, efficient, and end-to-end learning-based framework, namely AGD-Net. We first formulate the problem explicitly based on the classic gradient descent algorithm. AGD-Net can improve the reconstruction quality by more than 1.0 dB on average.
  arXiv  Detail & Related papers  (2021-08-12T05:54:09Z)
• LAPAR: Linearly-Assembled Pixel-Adaptive Regression Network for Single Image Super-Resolution and Beyond [75.37541439447314]
  Single image super-resolution (SISR) deals with a fundamental problem of upsampling a low-resolution (LR) image to its high-resolution (HR) version. This paper proposes a linearly-assembled pixel-adaptive regression network (LAPAR) to strike a sweet spot of deep model complexity and resulting SISR quality.
  arXiv  Detail & Related papers  (2021-05-21T15:47:18Z)
• A deep primal-dual proximal network for image restoration [8.797434238081372]
  We design a deep network, named DeepPDNet, built from primal-dual iterations associated with the minimization of a standard penalized likelihood with an analysis prior. Two different learning strategies: "Full learning" and "Partial learning" are proposed, the first one is the most efficient numerically. Extensive results show that the proposed DeepPDNet demonstrates excellent performance on the MNIST and the more complex BSD68, BSD100, and SET14 datasets for image restoration and single image super-resolution task.
  arXiv  Detail & Related papers  (2020-07-02T08:29:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.