FD$^2$: A Dedicated Framework for Fine-Grained Dataset Distillation

• URL: http://arxiv.org/abs/2603.25144v1
• Date: Thu, 26 Mar 2026 08:03:45 GMT
• Title: FD$^2$: A Dedicated Framework for Fine-Grained Dataset Distillation
• Abstract summary: We propose FD$2$, a framework for fine-grained dataset distillation.<n>During pretraining, a fine-grained characteristic constraint aligns each sample with its class prototype while repelling others.<n>Experiments on multiple fine-grained and general datasets show that FD$2$ integrates seamlessly with decoupled DD.
• Score: 46.27405453279254
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Dataset distillation (DD) compresses a large training set into a small synthetic set, reducing storage and training cost, and has shown strong results on general benchmarks. Decoupled DD further improves efficiency by splitting the pipeline into pretraining, sample distillation, and soft-label generation. However, existing decoupled methods largely rely on coarse class-label supervision and optimize samples within each class in a nearly identical manner. On fine-grained datasets, this often yields distilled samples that (i) retain large intra-class variation with subtle inter-class differences and (ii) become overly similar within the same class, limiting localized discriminative cues and hurting recognition. To solve the above-mentioned problems, we propose FD$^{2}$, a dedicated framework for Fine-grained Dataset Distillation. FD$^{2}$ localizes discriminative regions and constructs fine-grained representations for distillation. During pretraining, counterfactual attention learning aggregates discriminative representations to update class prototypes. During distillation, a fine-grained characteristic constraint aligns each sample with its class prototype while repelling others, and a similarity constraint diversifies attention across same-class samples. Experiments on multiple fine-grained and general datasets show that FD$^{2}$ integrates seamlessly with decoupled DD and improves performance in most settings, indicating strong transferability.

Related papers

• Dataset Distillation Based on Saliency-Driven Prototype Alignment [7.671427495273854]
  We propose a saliency-driven distillation framework that constructs class-discriminative latent prototypes to enhance representativeness and generalization.<n>The framework proceeds in two stages: (1) ensemble Grad-CAM++ saliency is used to construct prototypes emphasizing class-discriminative regions, and (2) hard-prototype refinement is then applied to construct challenging yet class-consistent prototypes.<n>Experiments across multiple benchmarks demonstrate consistent performance improvements over strong baselines.
  arXiv  Detail & Related papers  (2026-07-28T06:05:12Z)
• SAS: Semantic-aware Sampling for Generative Dataset Distillation [55.27114962330541]
  We introduce a semantic-aware perspective for dataset distillation by leveraging Contrastive Language-Image Pretraining (CLIP) as a semantic prior for post-sampling.<n>Our goal is to obtain distilled datasets that are not only compact but also semantically class-discriminative and diverse.
  arXiv  Detail & Related papers  (2026-05-18T08:05:46Z)
• Optimizing Distributional Geometry Alignment with Optimal Transport for Generative Dataset Distillation [109.13471554184554]
  We reformulate dataset distillation as an Optimal Transport (OT) distance minimization problem.<n>OT offers a geometrically faithful framework for distribution matching.<n>Our method consistently outperforms state-of-the-art approaches in an efficient manner.
  arXiv  Detail & Related papers  (2025-11-29T04:04:05Z)
• Rectified Decoupled Dataset Distillation: A Closer Look for Fair and Comprehensive Evaluation [36.444254126901065]
  We propose Rectified Decoupled dataset Distillation (RD$3$) to generate compact synthetic datasets.<n>RD$3$ provides a foundation for fair and reproducible comparisons in future dataset distillation research.
  arXiv  Detail & Related papers  (2025-09-24T03:47:04Z)
• AHDMIL: Asymmetric Hierarchical Distillation Multi-Instance Learning for Fast and Accurate Whole-Slide Image Classification [51.525891360380285]
  AHDMIL is an Asymmetric Hierarchical Distillation Multi-Instance Learning framework.<n>It eliminates irrelevant patches through a two-step training process.<n>It consistently outperforms previous state-of-the-art methods in both classification performance and inference speed.
  arXiv  Detail & Related papers  (2025-08-07T07:47:16Z)
• Enhancing Dataset Distillation via Non-Critical Region Refinement [29.858754062202213]
  We introduce the Non-Critical Region Refinement dataset Distillation (NRR-DD) method, which preserves instance-specific details and fine-grained regions in synthetic data.<n>We also present Distance-Based Representative (DBR) knowledge transfer, which eliminates the need for soft labels in training.<n> Experimental results show that NRR-DD achieves state-of-the-art performance on both small- and large-scale datasets.
  arXiv  Detail & Related papers  (2025-03-24T01:20:22Z)
• FedUV: Uniformity and Variance for Heterogeneous Federated Learning [5.9330433627374815]
  Federated learning is a promising framework to train neural networks with widely distributed data. Recent work has shown this is due to the final layer of the network being most prone to local bias. We investigate the training dynamics of the classifier by applying SVD to the weights motivated by the observation that freezing weights results in constant singular values.
  arXiv  Detail & Related papers  (2024-02-27T15:53:15Z)
• Latent Distribution Adjusting for Face Anti-Spoofing [29.204168516602568]
  We propose a unified framework called Latent Distribution Adjusting (LDA) to improve the robustness of the face anti-spoofing (FAS) model. To enhance the intra-class compactness and inter-class discrepancy, we propose a margin-based loss for providing distribution constrains for prototype learning. Our framework can 1) make the final representation space both intra-class compact and inter-class separable, 2) outperform the state-of-the-art methods on multiple standard FAS benchmarks.
  arXiv  Detail & Related papers  (2023-05-16T08:43:14Z)
• Intra-class Adaptive Augmentation with Neighbor Correction for Deep Metric Learning [99.14132861655223]
  We propose a novel intra-class adaptive augmentation (IAA) framework for deep metric learning. We reasonably estimate intra-class variations for every class and generate adaptive synthetic samples to support hard samples mining. Our method significantly improves and outperforms the state-of-the-art methods on retrieval performances by 3%-6%.
  arXiv  Detail & Related papers  (2022-11-29T14:52:38Z)
• Unsupervised Feature Learning by Cross-Level Instance-Group Discrimination [68.83098015578874]
  We integrate between-instance similarity into contrastive learning, not directly by instance grouping, but by cross-level discrimination. CLD effectively brings unsupervised learning closer to natural data and real-world applications. New state-of-the-art on self-supervision, semi-supervision, and transfer learning benchmarks, and beats MoCo v2 and SimCLR on every reported performance.
  arXiv  Detail & Related papers  (2020-08-09T21:13:13Z)
• Deep Semi-supervised Knowledge Distillation for Overlapping Cervical Cell Instance Segmentation [54.49894381464853]
  We propose to leverage both labeled and unlabeled data for instance segmentation with improved accuracy by knowledge distillation. We propose a novel Mask-guided Mean Teacher framework with Perturbation-sensitive Sample Mining. Experiments show that the proposed method improves the performance significantly compared with the supervised method learned from labeled data only.
  arXiv  Detail & Related papers  (2020-07-21T13:27:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.