IMS3: Breaking Distributional Aggregation in Diffusion-Based Dataset Distillation

• URL: http://arxiv.org/abs/2603.13960v1
• Date: Sat, 14 Mar 2026 14:15:03 GMT
• Title: IMS3: Breaking Distributional Aggregation in Diffusion-Based Dataset Distillation
• Abstract summary: Recently, diffusion-based dataset distillation methods have shown great promise by leveraging the strong generative capacity of diffusion models to produce diverse and structurally consistent samples.<n>Inversion-Matching (IM) introduces an inversion-guided fine-tuning process that aligns denoising trajectories with their inversion counterparts, broadening distributional coverage and enhancing diversity.<n> Selective Subgroup Sampling(S3) is a training-free sampling mechanism that improves inter-class separability by selecting synthetic subsets that are both representative and distinctive.
• Score: 41.83834685276882
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Dataset Distillation aims to synthesize compact datasets that can approximate the training efficacy of large-scale real datasets, offering an efficient solution to the increasing computational demands of modern deep learning. Recently, diffusion-based dataset distillation methods have shown great promise by leveraging the strong generative capacity of diffusion models to produce diverse and structurally consistent samples. However, a fundamental goal misalignment persists: diffusion models are optimized for generative likelihood rather than discriminative utility, resulting in over-concentration in high-density regions and inadequate coverage of boundary samples crucial for classification. To address this issue, we propose two complementary strategies. Inversion-Matching (IM) introduces an inversion-guided fine-tuning process that aligns denoising trajectories with their inversion counterparts, broadening distributional coverage and enhancing diversity. Selective Subgroup Sampling(S^3) is a training-free sampling mechanism that improves inter-class separability by selecting synthetic subsets that are both representative and distinctive. Extensive experiments demonstrate that our approach significantly enhances the discriminative quality and generalization of distilled datasets, achieving state-of-the-art performance among diffusion-based methods.

Related papers

• Geometry-Aware Dataset Condensation for Diffusion Model Training [103.45641113998839]
  We propose to reformulate real subset selection as a geometry-aware distribution alignment problem.<n>By incorporating one-sided partial optimal transport, our method selectively aligns a compact subset with the full data distribution.<n>Experiments across diffusion variants, subset sizes, image resolutions, and training rounds show that our method achieves superior fidelity and distributional coverage.
  arXiv  Detail & Related papers  (2026-06-04T08:53:58Z)
• SAS: Semantic-aware Sampling for Generative Dataset Distillation [55.27114962330541]
  We introduce a semantic-aware perspective for dataset distillation by leveraging Contrastive Language-Image Pretraining (CLIP) as a semantic prior for post-sampling.<n>Our goal is to obtain distilled datasets that are not only compact but also semantically class-discriminative and diverse.
  arXiv  Detail & Related papers  (2026-05-18T08:05:46Z)
• DMGD: Train-Free Dataset Distillation with Semantic-Distribution Matching in Diffusion Models [15.624945053686107]
  We propose a Dual Matching Guided Diffusion (DMGD) framework, centered on efficient training-free guidance.<n>We establish Semantic Matching via conditional likelihood optimization, eliminating the need for auxiliary classifiers.<n>We also propose a dynamic guidance mechanism that enhances the diversity of synthetic data while maintaining semantic alignment.
  arXiv  Detail & Related papers  (2026-05-05T15:37:50Z)
• Diffusion Models as Dataset Distillation Priors [39.4727398182562]
  We propose As Diffusion Priors (DAP), which formalizes representativeness by quantifying the similarity between synthetic and real data in feature space.<n>DAP outperforms state-of-the-art methods in generating high-fidelity datasets.<n>Our work establishes a theoretical connection between diffusion priors and the objectives of dataset distillation.
  arXiv  Detail & Related papers  (2025-10-20T11:04:09Z)
• Leveraging Diffusion Models for Synthetic Data Augmentation in Protein Subcellular Localization Classification [0.0]
  We implement a class-conditional denoising diffusion probabilistic model (DDPM) to produce label-consistent samples.<n>We explore their integration with real data via two hybrid training strategies: Mix Loss and Mix Representation.<n>Our findings highlight the importance of realistic data generation and robust supervision when incorporating generative augmentation into biomedical image classification.
  arXiv  Detail & Related papers  (2025-05-28T22:58:50Z)
• Diversity-Driven Generative Dataset Distillation Based on Diffusion Model with Self-Adaptive Memory [33.38900857290244]
  We present a diversity-driven generative dataset distillation method based on a diffusion model to solve this problem.<n>We introduce self-adaptive memory to align the distribution between distilled and real datasets, assessing the representativeness.<n>Our method outperforms existing state-of-the-art methods in most situations.
  arXiv  Detail & Related papers  (2025-05-26T03:48:56Z)
• Taming Diffusion for Dataset Distillation with High Representativeness [49.3818035378669]
  D3HR is a novel diffusion-based framework to generate distilled datasets with high representativeness.<n>Our experiments demonstrate that D3HR can achieve higher accuracy across different model architectures.
  arXiv  Detail & Related papers  (2025-05-23T22:05:59Z)
• Constrained Diffusion Models via Dual Training [80.03953599062365]
  Diffusion processes are prone to generating samples that reflect biases in a training dataset. We develop constrained diffusion models by imposing diffusion constraints based on desired distributions. We show that our constrained diffusion models generate new data from a mixture data distribution that achieves the optimal trade-off among objective and constraints.
  arXiv  Detail & Related papers  (2024-08-27T14:25:42Z)
• Diffusion Models as Constrained Samplers for Optimization with Unknown Constraints [55.39203337683045]
  We propose to perform optimization within the data manifold using diffusion models.<n>Depending on the differentiability of the objective function, we propose two different sampling methods.<n>Our method achieves better or comparable performance with previous state-of-the-art baselines.
  arXiv  Detail & Related papers  (2024-02-28T03:09:12Z)
• M3D: Dataset Condensation by Minimizing Maximum Mean Discrepancy [26.227927019615446]
  Training state-of-the-art (SOTA) deep models often requires extensive data, resulting in substantial training and storage costs. dataset condensation has been developed to learn a small synthetic set that preserves essential information from the original large-scale dataset. We present a novel DM-based method named M3D for dataset condensation by Minimizing the Maximum Mean Discrepancy.
  arXiv  Detail & Related papers  (2023-12-26T07:45:32Z)
• Dataset Distillation via the Wasserstein Metric [34.06251608504682]
  We introduce WMDD (Wasserstein Metric-based dataset Distillation), a straightforward yet powerful method that employs the Wasserstein metric to enhance distribution matching.<n>Our experiments demonstrate WMDD's effectiveness and adaptability, highlighting its potential for advancing machine learning applications at scale.
  arXiv  Detail & Related papers  (2023-11-30T13:15:28Z)
• CAFE: Learning to Condense Dataset by Aligning Features [72.99394941348757]
  We propose a novel scheme to Condense dataset by Aligning FEatures (CAFE) At the heart of our approach is an effective strategy to align features from the real and synthetic data across various scales. We validate the proposed CAFE across various datasets, and demonstrate that it generally outperforms the state of the art.
  arXiv  Detail & Related papers  (2022-03-03T05:58:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.