Dataset Distillation Based on Saliency-Driven Prototype Alignment

• URL: http://arxiv.org/abs/2607.25318v2
• Date: Fri, 31 Jul 2026 07:43:16 GMT
• Title: Dataset Distillation Based on Saliency-Driven Prototype Alignment
• Abstract summary: We propose a saliency-driven distillation framework that constructs class-discriminative latent prototypes to enhance representativeness and generalization.<n>The framework proceeds in two stages: (1) ensemble Grad-CAM++ saliency is used to construct prototypes emphasizing class-discriminative regions, and (2) hard-prototype refinement is then applied to construct challenging yet class-consistent prototypes.<n>Experiments across multiple benchmarks demonstrate consistent performance improvements over strong baselines.
• Score: 7.671427495273854
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Dataset distillation aims to synthesize compact datasets that can approximate the performance of full-data training while significantly reducing computational and storage costs. However, diffusion-based distillation methods often struggle to preserve structural coherence and generalization, especially in visually complex domains. This issue often stems from latent prototypes that are weakly aligned with class-discriminative regions and contaminated by irrelevant background, thereby degrading generation quality and generalization. To address this limitation, we propose a saliency-driven distillation framework that constructs class-discriminative latent prototypes to enhance representativeness and generalization. The framework proceeds in two stages: (1) ensemble Grad-CAM++ saliency is used to construct prototypes emphasizing class-discriminative regions, and (2) hard-prototype refinement is then applied to construct challenging yet class-consistent prototypes, thereby enhancing discriminability and diversity. Importantly, the diffusion backbones (e.g., LDM and DiT) remain frozen; only lightweight classifiers used for saliency extraction are trained. Extensive experiments across multiple benchmarks demonstrate consistent performance improvements over strong baselines. Code will be released.

Related papers

• SWARD: Stochastic Window-Attention-Based Relational Distillation for Cross-Architectural Semantic Segmentation [1.1617094867403568]
  Large-scale vision foundation models have driven substantial gains on dense prediction tasks such as semantic segmentation.<n>Modern foundation teachers are predominantly transformer-based that encode global context, whereas efficient students are typically convolutional networks with locally biased receptive fields.<n>Existing distillation methods largely assume architectural homogeneity and rely on direct feature mimicry, which fails to bridge this representational gap.<n>We propose SWARD, a knowledge distillation framework that addresses this gap through two complementary mechanisms.
  arXiv  Detail & Related papers  (2026-05-31T04:29:31Z)
• SAS: Semantic-aware Sampling for Generative Dataset Distillation [55.27114962330541]
  We introduce a semantic-aware perspective for dataset distillation by leveraging Contrastive Language-Image Pretraining (CLIP) as a semantic prior for post-sampling.<n>Our goal is to obtain distilled datasets that are not only compact but also semantically class-discriminative and diverse.
  arXiv  Detail & Related papers  (2026-05-18T08:05:46Z)
• FD$^2$: A Dedicated Framework for Fine-Grained Dataset Distillation [46.27405453279254]
  We propose FD$2$, a framework for fine-grained dataset distillation.<n>During pretraining, a fine-grained characteristic constraint aligns each sample with its class prototype while repelling others.<n>Experiments on multiple fine-grained and general datasets show that FD$2$ integrates seamlessly with decoupled DD.
  arXiv  Detail & Related papers  (2026-03-26T08:03:45Z)
• IMS3: Breaking Distributional Aggregation in Diffusion-Based Dataset Distillation [41.83834685276882]
  Recently, diffusion-based dataset distillation methods have shown great promise by leveraging the strong generative capacity of diffusion models to produce diverse and structurally consistent samples.<n>Inversion-Matching (IM) introduces an inversion-guided fine-tuning process that aligns denoising trajectories with their inversion counterparts, broadening distributional coverage and enhancing diversity.<n> Selective Subgroup Sampling(S3) is a training-free sampling mechanism that improves inter-class separability by selecting synthetic subsets that are both representative and distinctive.
  arXiv  Detail & Related papers  (2026-03-14T14:15:03Z)
• Generalizable Knowledge Distillation from Vision Foundation Models for Semantic Segmentation [73.32435804067883]
  Generalizable Knowledge Distillation (GKD) is a multi-stage framework that explicitly enhances generalization.<n>Experiments on five domain generalization benchmarks demonstrate that GKD consistently outperforms existing KD methods.
  arXiv  Detail & Related papers  (2026-03-03T03:18:12Z)
• ManifoldGD: Training-Free Hierarchical Manifold Guidance for Diffusion-Based Dataset Distillation [9.230247128710865]
  We propose a training-free diffusion-based framework that integrates manifold consistent guidance at every denoising timestep.<n>ManifoldGD improves representativeness, diversity, and image fidelity without requiring any model retraining.
  arXiv  Detail & Related papers  (2026-02-26T18:07:10Z)
• Utility Boundary of Dataset Distillation: Scaling and Configuration-Coverage Laws [6.172966466468818]
  It is unclear under what conditions distilled data can retain the effectiveness of full datasets when the training configuration changes.<n>We propose a unified theoretical framework, termed configuration--configuration-error analysis, which reformulates major DD approaches under a common generalization-error perspective.<n>Our analysis reveals that various matching methods are interchangeable surrogates, reducing the same generalization error, clarifying why they can all achieve dataset distillation.
  arXiv  Detail & Related papers  (2025-12-05T15:37:38Z)
• Divide, Conquer and Unite: Hierarchical Style-Recalibrated Prototype Alignment for Federated Medical Image Segmentation [66.82598255715696]
  Federated learning enables multiple medical institutions to train a global model without sharing data.<n>Current approaches primarily focus on final-layer features, overlooking critical multi-level cues.<n>We propose FedBCS to bridge feature representation gaps via domain-invariant contextual prototypes alignment.
  arXiv  Detail & Related papers  (2025-11-14T04:15:34Z)
• Diffusion Models as Dataset Distillation Priors [39.4727398182562]
  We propose As Diffusion Priors (DAP), which formalizes representativeness by quantifying the similarity between synthetic and real data in feature space.<n>DAP outperforms state-of-the-art methods in generating high-fidelity datasets.<n>Our work establishes a theoretical connection between diffusion priors and the objectives of dataset distillation.
  arXiv  Detail & Related papers  (2025-10-20T11:04:09Z)
• Learning Robust Diffusion Models from Imprecise Supervision [75.53546939251146]
  DMIS is a unified framework for training robust Conditional Diffusion Models from Imprecise Supervision.<n>Our framework is derived from likelihood and decomposes the objective into generative and classification components.<n>Experiments on diverse forms of imprecise supervision, covering tasks covering image generation, weakly supervised learning, and dataset condensation demonstrate that DMIS consistently produces high-quality and class-discriminative samples.
  arXiv  Detail & Related papers  (2025-10-03T14:00:32Z)
• Restoration Score Distillation: From Corrupted Diffusion Pretraining to One-Step High-Quality Generation [82.39763984380625]
  We propose textitRestoration Score Distillation (RSD), a principled generalization of Denoising Score Distillation (DSD)<n>RSD accommodates a broader range of corruption types, such as blurred, incomplete, or low-resolution images.<n>It consistently surpasses its teacher model across diverse restoration tasks on both natural and scientific datasets.
  arXiv  Detail & Related papers  (2025-05-19T17:21:03Z)
• GCC: Generative Calibration Clustering [55.44944397168619]
  We propose a novel Generative Clustering (GCC) method to incorporate feature learning and augmentation into clustering procedure. First, we develop a discrimirative feature alignment mechanism to discover intrinsic relationship across real and generated samples. Second, we design a self-supervised metric learning to generate more reliable cluster assignment.
  arXiv  Detail & Related papers  (2024-04-14T01:51:11Z)
• Learning Invariant Molecular Representation in Latent Discrete Space [52.13724532622099]
  We propose a new framework for learning molecular representations that exhibit invariance and robustness against distribution shifts. Our model achieves stronger generalization against state-of-the-art baselines in the presence of various distribution shifts.
  arXiv  Detail & Related papers  (2023-10-22T04:06:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.