DIVER:Diving Deeper into Distilled Data via Expressive Semantic Recovery

• URL: http://arxiv.org/abs/2605.12649v1
• Date: Tue, 12 May 2026 18:55:53 GMT
• Title: DIVER:Diving Deeper into Distilled Data via Expressive Semantic Recovery
• Abstract summary: We propose a novel dual-stage distillation framework called $textbfDIVER$.<n>Semantic inheritance distills high-level semantics of abstract distilled images into the latent space.<n>Semantic guidance improves the preservation of the original semantics.<n>Semantic fusion is designed to provide semantic guidance only during the concrete phase of the reverse process.
• Score: 13.429822059631704
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Dataset distillation aims to synthesize a compact proxy dataset that is unreadable or non-raw from the original dataset for privacy protection and highly efficient learning. However, previous approaches typically adopt a single-stage distillation paradigm, which suffers from learning specific patterns that overfit on a prior architecture, consequently suppressing the expression of semantics and leading to performance degradation across heterogeneous architectures. To address this issue, we propose a novel dual-stage distillation framework called ${\textbf{DIVER}}$, which leverages the pre-trained diffusion model to dive deeper into $\textbf{DI}$stilled data $\textbf{V}$ia $\textbf{E}$xpressive semantic $\textbf{R}$ecovery, an entire process of semantic inheritance, guidance, and fusion. Semantic inheritance distills high-level semantics of abstract distilled images into the latent space to filter out architecture-specific ``noise" and retain the intrinsic semantics. Furthermore, semantic guidance improves the preservation of the original semantics by directing the reverse procedure. Finally, semantic fusion is designed to provide semantic guidance only during the concrete phase of the reverse process, preventing semantic ambiguity and artifacts while maintaining the guidance information. Extensive experiments validate the effectiveness and efficiency of DIVER in improving classical distillation techniques and significantly improving cross-architecture generalization, requiring processing time comparable to raw DiT on ImageNet (256$\times$256) with only 4 GB of GPU memory usage. Code is available: https://github.com/einsteinxia/DIVER.

Related papers

• SAS: Semantic-aware Sampling for Generative Dataset Distillation [55.27114962330541]
  We introduce a semantic-aware perspective for dataset distillation by leveraging Contrastive Language-Image Pretraining (CLIP) as a semantic prior for post-sampling.<n>Our goal is to obtain distilled datasets that are not only compact but also semantically class-discriminative and diverse.
  arXiv  Detail & Related papers  (2026-05-18T08:05:46Z)
• Where Do Flow Semantics Reside? A Protocol-Native Tabular Pretraining Paradigm for Encrypted Traffic Classification [5.044786941116112]
  Self-supervised masked modeling shows promise for encrypted traffic classification by masking and reconstructing raw bytes.<n>Recent work reveals these methods fail to reduce reliance on labeled data despite costly pretraining.<n>We argue the root cause is inductive bias mismatch: flattening traffic into sequences byte destroys protocol-defined semantics.
  arXiv  Detail & Related papers  (2026-03-09T15:15:23Z)
• RecTok: Reconstruction Distillation along Rectified Flow [85.51292475005151]
  We propose RecTok, which overcomes the limitations of high-dimensional visual tokenizers through two key innovations.<n>Our method distills the semantic information in VFMs into the forward flow trajectories in flow matching.<n>Our RecTok achieves superior image reconstruction, generation quality, and discriminative performance.
  arXiv  Detail & Related papers  (2025-12-15T15:14:20Z)
• Unleashing Degradation-Carrying Features in Symmetric U-Net: Simpler and Stronger Baselines for All-in-One Image Restoration [52.82397287366076]
  All-in-one image restoration aims to handle diverse degradations (e.g., noise, blur, adverse weather) within a unified framework.<n>In this work, we reveal a critical insight: well-crafted feature extraction inherently encodes degradation-carrying information.<n>Our symmetric design preserves intrinsic degradation signals robustly, rendering simple additive fusion in skip connections.
  arXiv  Detail & Related papers  (2025-12-11T12:20:31Z)
• Semantics Lead the Way: Harmonizing Semantic and Texture Modeling with Asynchronous Latent Diffusion [59.30874672131483]
  Latent Diffusion Models (LDMs) inherently follow a coarse-to-fine generation process.<n>Recent advances have integrated semantic priors from pretrained visual encoders to further enhance LDMs.<n>We propose Semantic-First Diffusion (SFD), a latent diffusion paradigm that explicitly prioritizes semantic formation.
  arXiv  Detail & Related papers  (2025-12-04T15:57:27Z)
• EDITS: Enhancing Dataset Distillation with Implicit Textual Semantics [12.818622596576775]
  EDITS is a novel framework that exploits the implicit textual semantics within the image data to achieve enhanced distillation.<n>In this paper, we propose EDITS, a novel framework that exploits the implicit textual semantics within the image data to achieve enhanced distillation.
  arXiv  Detail & Related papers  (2025-09-17T09:48:39Z)
• Towards Generalized and Training-Free Text-Guided Semantic Manipulation [123.80467566483038]
  Text-guided semantic manipulation refers to semantically editing an image generated from a source prompt to match a target prompt.<n>We propose a novel $textitGTF$ for text-guided semantic manipulation, which has the following attractive capabilities.<n>Our experiments demonstrate the efficacy of our approach, highlighting its potential to advance the state-of-the-art in semantics manipulation.
  arXiv  Detail & Related papers  (2025-04-24T05:54:56Z)
• D$^4$M: Dataset Distillation via Disentangled Diffusion Model [4.568710926635445]
  We propose an efficient framework for dataset distillation via Disentangled Diffusion Model (D$4$M) Compared to architecture-dependent methods, D$4$M employs latent diffusion model to guarantee consistency and incorporates label information into category prototypes. D$4$M demonstrates superior performance and robust generalization, surpassing the SOTA methods across most aspects.
  arXiv  Detail & Related papers  (2024-07-21T12:16:20Z)
• Spatial Semantic Recurrent Mining for Referring Image Segmentation [63.34997546393106]
  We propose Stextsuperscript2RM to achieve high-quality cross-modality fusion. It follows a working strategy of trilogy: distributing language feature, spatial semantic recurrent coparsing, and parsed-semantic balancing. Our proposed method performs favorably against other state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2024-05-15T00:17:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.