Geometry-Aware Dataset Condensation for Diffusion Model Training

• URL: http://arxiv.org/abs/2606.05883v1
• Date: Thu, 04 Jun 2026 08:53:58 GMT
• Title: Geometry-Aware Dataset Condensation for Diffusion Model Training
• Abstract summary: We propose to reformulate real subset selection as a geometry-aware distribution alignment problem.<n>By incorporating one-sided partial optimal transport, our method selectively aligns a compact subset with the full data distribution.<n>Experiments across diffusion variants, subset sizes, image resolutions, and training rounds show that our method achieves superior fidelity and distributional coverage.
• Score: 103.45641113998839
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Dataset condensation aims to construct compact datasets from real data via synthesis or selection. However, existing approaches are ill-suited for diffusion model training: synthetic data generation often yields low-fidelity samples unsuitable for authentic modeling, while real subset selection typically fails to preserve the distributional geometry required by diffusion likelihood objectives. To address this, we propose to reformulate real subset selection as a geometry-aware distribution alignment problem. By incorporating one-sided partial optimal transport, our method selectively aligns a compact subset with the full data distribution while allowing unmatched mass in low-density regions, ensuring the preserved geometric structure necessary for effective diffusion model training. To further ensure distributional fidelity, we complement geometric alignment with lightweight feature-statistics and semantic consistency regularization. An efficient two-stage discrete optimization strategy is proposed to achieve this alignment objective. Extensive experiments across diffusion variants, subset sizes, image resolutions, and training rounds show that our method achieves superior fidelity and distributional coverage in diffusion model training. Codes are available at https://github.com/2018cx/GADC.

Related papers

• Variational Template Matching with Statistical Fusion for Anomaly Detection in Patterned Structures [5.22657564684602]
  Anomaly detection in structured images is challenging in small-data settings where deep learning approaches are costly or impractical.<n>We propose a variational template matching framework that represents anomaly templates as a family of transformed instances.<n>We also introduce a density-based statistical anomaly score derived from local intensity distributions.
  arXiv  Detail & Related papers  (2026-09-10T00:48:35Z)
• IMS3: Breaking Distributional Aggregation in Diffusion-Based Dataset Distillation [41.83834685276882]
  Recently, diffusion-based dataset distillation methods have shown great promise by leveraging the strong generative capacity of diffusion models to produce diverse and structurally consistent samples.<n>Inversion-Matching (IM) introduces an inversion-guided fine-tuning process that aligns denoising trajectories with their inversion counterparts, broadening distributional coverage and enhancing diversity.<n> Selective Subgroup Sampling(S3) is a training-free sampling mechanism that improves inter-class separability by selecting synthetic subsets that are both representative and distinctive.
  arXiv  Detail & Related papers  (2026-03-14T14:15:03Z)
• Flow Matching is Adaptive to Manifold Structures [32.55405572762157]
  Flow matching is a simulation-based alternative to diffusion-based generative modeling.<n>We show how flow matching adapts to data geometry and circumvents the curse of dimensionality.
  arXiv  Detail & Related papers  (2026-02-25T23:52:32Z)
• GeoDM: Geometry-aware Distribution Matching for Dataset Distillation [5.993128231927707]
  We propose a geometry-aware distribution-matching framework, called textbfGeoDM.<n>To adapt to the underlying data geometry, we introduce learnable curvature and weight parameters for three kinds of geometries.<n>Our theoretical analysis shows that the geometry-aware distribution matching in a product space yields a smaller generalization error bound than the Euclidean counterparts.
  arXiv  Detail & Related papers  (2025-12-09T07:31:57Z)
• Optimizing Distributional Geometry Alignment with Optimal Transport for Generative Dataset Distillation [109.13471554184554]
  We reformulate dataset distillation as an Optimal Transport (OT) distance minimization problem.<n>OT offers a geometrically faithful framework for distribution matching.<n>Our method consistently outperforms state-of-the-art approaches in an efficient manner.
  arXiv  Detail & Related papers  (2025-11-29T04:04:05Z)
• Composition and Alignment of Diffusion Models using Constrained Learning [79.36736636241564]
  Diffusion models have become prevalent in generative modeling due to their ability to sample from complex distributions.<n>Two commonly used methods are: (i) alignment, which involves fine-tuning a diffusion model to align it with a reward; and (ii) composition, which combines several pre-trained diffusion models, each emphasizing a desirable attribute in the generated outputs.<n>We propose a constrained optimization framework that unifies alignment and composition of diffusion models by enforcing that the aligned model satisfies reward constraints and/or remains close to (potentially multiple) pre-trained models.
  arXiv  Detail & Related papers  (2025-08-26T15:06:30Z)
• Constrained Diffusion Models via Dual Training [80.03953599062365]
  Diffusion processes are prone to generating samples that reflect biases in a training dataset. We develop constrained diffusion models by imposing diffusion constraints based on desired distributions. We show that our constrained diffusion models generate new data from a mixture data distribution that achieves the optimal trade-off among objective and constraints.
  arXiv  Detail & Related papers  (2024-08-27T14:25:42Z)
• Improved Distribution Matching for Dataset Condensation [91.55972945798531]
  We propose a novel dataset condensation method based on distribution matching. Our simple yet effective method outperforms most previous optimization-oriented methods with much fewer computational resources.
  arXiv  Detail & Related papers  (2023-07-19T04:07:33Z)
• Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Data [68.62134204367668]
  This paper studies score approximation, estimation, and distribution recovery of diffusion models, when data are supported on an unknown low-dimensional linear subspace. We show that with a properly chosen neural network architecture, the score function can be both accurately approximated and efficiently estimated. The generated distribution based on the estimated score function captures the data geometric structures and converges to a close vicinity of the data distribution.
  arXiv  Detail & Related papers  (2023-02-14T17:02:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.