Enhancing Dataset Distillation via Non-Critical Region Refinement

• URL: http://arxiv.org/abs/2503.18267v1
• Date: Mon, 24 Mar 2025 01:20:22 GMT
• Title: Enhancing Dataset Distillation via Non-Critical Region Refinement
• Authors: Minh-Tuan Tran, Trung Le, Xuan-May Le, Thanh-Toan Do, Dinh Phung,
• Abstract summary: We introduce the Non-Critical Region Refinement dataset Distillation (NRR-DD) method, which preserves instance-specific details and fine-grained regions in synthetic data.<n>We also present Distance-Based Representative (DBR) knowledge transfer, which eliminates the need for soft labels in training.<n> Experimental results show that NRR-DD achieves state-of-the-art performance on both small- and large-scale datasets.
• Score: 29.858754062202213
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Dataset distillation has become a popular method for compressing large datasets into smaller, more efficient representations while preserving critical information for model training. Data features are broadly categorized into two types: instance-specific features, which capture unique, fine-grained details of individual examples, and class-general features, which represent shared, broad patterns across a class. However, previous approaches often struggle to balance these features-some focus solely on class-general patterns, neglecting finer instance details, while others prioritize instance-specific features, overlooking the shared characteristics essential for class-level understanding. In this paper, we introduce the Non-Critical Region Refinement Dataset Distillation (NRR-DD) method, which preserves instance-specific details and fine-grained regions in synthetic data while enriching non-critical regions with class-general information. This approach enables models to leverage all pixel information, capturing both feature types and enhancing overall performance. Additionally, we present Distance-Based Representative (DBR) knowledge transfer, which eliminates the need for soft labels in training by relying on the distance between synthetic data predictions and one-hot encoded labels. Experimental results show that NRR-DD achieves state-of-the-art performance on both small- and large-scale datasets. Furthermore, by storing only two distances per instance, our method delivers comparable results across various settings. The code is available at https://github.com/tmtuan1307/NRR-DD.

Related papers

• Scalable Graph Attention-based Instance Selection via Mini-Batch Sampling and Hierarchical Hashing [0.24578723416255752]
  Instance selection (IS) is important in machine learning for reducing dataset size while keeping key characteristics. This paper introduces a graph attention-based instance selection (GAIS) method that uses attention mechanisms to identify informative instances. We present two approaches for scalable graph construction: a distance-based mini-batch sampling technique that reduces through strategic batch processing, and a hierarchical hashing approach that allows for efficient similarity through random projections.
  arXiv  Detail & Related papers  (2025-02-27T17:17:53Z)
• DatasetDM: Synthesizing Data with Perception Annotations Using Diffusion Models [61.906934570771256]
  We present a generic dataset generation model that can produce diverse synthetic images and perception annotations. Our method builds upon the pre-trained diffusion model and extends text-guided image synthesis to perception data generation. We show that the rich latent code of the diffusion model can be effectively decoded as accurate perception annotations using a decoder module.
  arXiv  Detail & Related papers  (2023-08-11T14:38:11Z)
• infoVerse: A Universal Framework for Dataset Characterization with Multidimensional Meta-information [68.76707843019886]
  infoVerse is a universal framework for dataset characterization. infoVerse captures multidimensional characteristics of datasets by incorporating various model-driven meta-information. In three real-world applications (data pruning, active learning, and data annotation), the samples chosen on infoVerse space consistently outperform strong baselines.
  arXiv  Detail & Related papers  (2023-05-30T18:12:48Z)
• CHALLENGER: Training with Attribution Maps [63.736435657236505]
  We show that utilizing attribution maps for training neural networks can improve regularization of models and thus increase performance. In particular, we show that our generic domain-independent approach yields state-of-the-art results in vision, natural language processing and on time series tasks.
  arXiv  Detail & Related papers  (2022-05-30T13:34:46Z)
• Reprint: a randomized extrapolation based on principal components for data augmentation [19.797216197418926]
  This paper presents a simple and effective hidden-space data augmentation method for imbalanced data classification.<n>Given hidden-space representations of samples in each class, REPRINT extrapolates, in a randomized fashion, augmented examples for target class.<n>This method involves a label refinement component which allows to synthesize new soft labels for augmented examples.
  arXiv  Detail & Related papers  (2022-04-26T01:38:47Z)
• Improving Contrastive Learning on Imbalanced Seed Data via Open-World Sampling [96.8742582581744]
  We present an open-world unlabeled data sampling framework called Model-Aware K-center (MAK) MAK follows three simple principles: tailness, proximity, and diversity. We demonstrate that MAK can consistently improve both the overall representation quality and the class balancedness of the learned features.
  arXiv  Detail & Related papers  (2021-11-01T15:09:41Z)
• Multi-dataset Pretraining: A Unified Model for Semantic Segmentation [97.61605021985062]
  We propose a unified framework, termed as Multi-Dataset Pretraining, to take full advantage of the fragmented annotations of different datasets. This is achieved by first pretraining the network via the proposed pixel-to-prototype contrastive loss over multiple datasets. In order to better model the relationship among images and classes from different datasets, we extend the pixel level embeddings via cross dataset mixing.
  arXiv  Detail & Related papers  (2021-06-08T06:13:11Z)
• Adaptive Prototypical Networks with Label Words and Joint Representation Learning for Few-Shot Relation Classification [17.237331828747006]
  This work focuses on few-shot relation classification (FSRC) We propose an adaptive mixture mechanism to add label words to the representation of the class prototype. Experiments have been conducted on FewRel under different few-shot (FS) settings.
  arXiv  Detail & Related papers  (2021-01-10T11:25:42Z)
• Selecting Relevant Features from a Multi-domain Representation for Few-shot Classification [91.67977602992657]
  We propose a new strategy based on feature selection, which is both simpler and more effective than previous feature adaptation approaches. We show that a simple non-parametric classifier built on top of such features produces high accuracy and generalizes to domains never seen during training.
  arXiv  Detail & Related papers  (2020-03-20T15:44:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.