Self-Supervised Representation-Guided Generative Dataset Distillation

• URL: http://arxiv.org/abs/2608.03218v1
• Date: Tue, 04 Aug 2026 06:51:45 GMT
• Title: Self-Supervised Representation-Guided Generative Dataset Distillation
• Abstract summary: We propose self-supervised representation-guided generative dataset distillation (SRG)<n>SRG translates the SSL geometry into diffusion guidance.<n>It consistently outperforms evaluated generative baselines across multiple datasets and IPC settings.
• Score: 55.27114962330541
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Dataset distillation compresses a large training set into a compact synthetic set while retaining its downstream utility. Most existing methods target randomly initialized networks, whereas modern vision systems often adapt frozen pretrained encoders with lightweight modules. Distilled samples should therefore preserve the discriminative geometry of the pretrained representation space, which existing generative objectives do not explicitly consider. We propose self-supervised representation-guided generative dataset distillation (SRG), a framework that translates the SSL geometry into diffusion guidance. Specifically, SRG constructs class-wise prototypes from real-image SSL representations and performs guidance through three SSL-space objectives for prototype alignment, inter-class discrimination, and intra-class assignment. During diffusion sampling, it adopts a stage-wise guidance strategy: early denoising is anchored to the latent of the real image whose SSL representation is nearest to the assigned prototype, whereas later denoising is guided by the SSL-space objectives. This division preserves the visual realism provided by the generative prior while progressively steering samples toward representative and class-discriminative regions of the SSL representation space. SRG consistently outperforms the evaluated generative baselines across multiple datasets and IPC settings. A cross-encoder evaluation further indicates transfer across pretrained representation spaces. These results demonstrate the effectiveness of representation-guided generation for dataset distillation with pretrained SSL models.

Related papers

• SSC-Priors: Exploring Semantic and Visibility Priors to Boost Lidar Semantic Scene Completion [61.665449818739155]
  This paper investigates easy strategies to boost the performance of existing networks for lidar semantic scene completion (SSC)<n>We use semantic pseudo-labels and sensor visibility information to improve existing approaches.<n>We show that endowing input point clouds with semantic pseudo-labels from off-the-shelf segmenters significantly improves the performance of existing SSC models.
  arXiv  Detail & Related papers  (2026-09-15T16:37:46Z)
• Probing Geospatial SSL Representations with Environmental Signals [51.908606610432194]
  Self-supervised learning (SSL) is designed to learn generic, transferable representations rather than representations optimized for a single task.<n>We ask a different question: do SSL representations preserve statistical associations with environmental variables that co-vary with the imaging process?<n>To answer this question, we probe SSL representations using co-located ERA5 reanalysis variables, a global dataset of physically consistent environmental variables.
  arXiv  Detail & Related papers  (2026-07-06T15:24:25Z)
• Generate in Reconstruction Space, Match in Semantic Space: Transport Geometry for One-Step Generation [16.138914735031744]
  Generative modeling and self-supervised representation learning (SSL) optimize structurally different objectives.<n>We study the benefits of SSL in generative modeling in the framework of one-step generation.<n>We find that this objective becomes highly effective when computed in a semantically structured SSL feature space.
  arXiv  Detail & Related papers  (2026-05-30T04:03:47Z)
• Quantifying the Pre-training Dividend: Generative versus Latent Self-Supervised Learning for Time Series Foundation Models [5.3565231231826695]
  Self-supervised learning (SSL) in vision and NLP has motivated its rapid adoption for time series.<n>We establish a framework to evaluate the "pre-training dividend": the value added by SSL across diverse temporal tasks.<n>Our analysis reveals that the pre-training dividend is highly asymmetric. SSL yields gains of up to 375% for anomaly detection and classification, yet remains marginal for forecasting.
  arXiv  Detail & Related papers  (2026-05-19T07:13:20Z)
• Enhancing Semantic Segmentation with Continual Self-Supervised Pre-training [11.897717409259492]
  Self-supervised learning (SSL) has emerged as a central paradigm for training foundation models.<n>We propose GLARE, a novel continual self-supervised pre-training task designed to enhance downstream segmentation performance.
  arXiv  Detail & Related papers  (2025-09-22T14:11:02Z)
• Deciphering the Projection Head: Representation Evaluation Self-supervised Learning [6.375931203397043]
  Self-supervised learning (SSL) aims to learn intrinsic features without labels. Projection head always plays an important role in improving the performance of the downstream task. We propose a Representation Evaluation Design (RED) in SSL models in which a shortcut connection between the representation and the projection vectors is built.
  arXiv  Detail & Related papers  (2023-01-28T13:13:53Z)
• Improving Self-Supervised Learning by Characterizing Idealized Representations [155.1457170539049]
  We prove necessary and sufficient conditions for any task invariant to given data augmentations. For contrastive learning, our framework prescribes simple but significant improvements to previous methods. For non-contrastive learning, we use our framework to derive a simple and novel objective.
  arXiv  Detail & Related papers  (2022-09-13T18:01:03Z)
• Structure-Aware Feature Generation for Zero-Shot Learning [108.76968151682621]
  We introduce a novel structure-aware feature generation scheme, termed as SA-GAN, to account for the topological structure in learning both the latent space and the generative networks. Our method significantly enhances the generalization capability on unseen-classes and consequently improve the classification performance.
  arXiv  Detail & Related papers  (2021-08-16T11:52:08Z)
• Trash to Treasure: Harvesting OOD Data with Cross-Modal Matching for Open-Set Semi-Supervised Learning [101.28281124670647]
  Open-set semi-supervised learning (open-set SSL) investigates a challenging but practical scenario where out-of-distribution (OOD) samples are contained in the unlabeled data. We propose a novel training mechanism that could effectively exploit the presence of OOD data for enhanced feature learning. Our approach substantially lifts the performance on open-set SSL and outperforms the state-of-the-art by a large margin.
  arXiv  Detail & Related papers  (2021-08-12T09:14:44Z)
• Closed-Form Factorization of Latent Semantics in GANs [65.42778970898534]
  A rich set of interpretable dimensions has been shown to emerge in the latent space of the Generative Adversarial Networks (GANs) trained for synthesizing images. In this work, we examine the internal representation learned by GANs to reveal the underlying variation factors in an unsupervised manner. We propose a closed-form factorization algorithm for latent semantic discovery by directly decomposing the pre-trained weights.
  arXiv  Detail & Related papers  (2020-07-13T18:05:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.