Beyond Pixels: Efficient Dataset Distillation via Sparse Gaussian Representation

• URL: http://arxiv.org/abs/2509.26219v1
• Date: Tue, 30 Sep 2025 13:19:05 GMT
• Title: Beyond Pixels: Efficient Dataset Distillation via Sparse Gaussian Representation
• Authors: Chenyang Jiang, Zhengcen Li, Hang Zhao, Qiben Shan, Shaocong Wu, Jingyong Su,
• Abstract summary: We propose GSDD, a novel and efficient sparse representation for dataset distillation based on 2D Gaussians.<n>Instead of representing all pixels equally, GSDD encodes critical discnative information in a distilled image using only small number of Gaussian primitives.<n>Experiments show that GSDD achieves state-of-the-art performance on CIFAR-10, CIFAR-100, and ImageNet subsets, while remaining highly efficient encoding and decoding cost.
• Score: 37.57424511974552
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Dataset distillation has emerged as a promising paradigm that synthesizes compact, informative datasets capable of retaining the knowledge of large-scale counterparts, thereby addressing the substantial computational and storage burdens of modern model training. Conventional approaches typically rely on dense pixel-level representations, which introduce redundancy and are difficult to scale up. In this work, we propose GSDD, a novel and efficient sparse representation for dataset distillation based on 2D Gaussians. Instead of representing all pixels equally, GSDD encodes critical discriminative information in a distilled image using only a small number of Gaussian primitives. This sparse representation could improve dataset diversity under the same storage budget, enhancing coverage of difficult samples and boosting distillation performance. To ensure both efficiency and scalability, we adapt CUDA-based splatting operators for parallel inference and training, enabling high-quality rendering with minimal computational and memory overhead. Our method is simple yet effective, broadly applicable to different distillation pipelines, and highly scalable. Experiments show that GSDD achieves state-of-the-art performance on CIFAR-10, CIFAR-100, and ImageNet subsets, while remaining highly efficient encoding and decoding cost. Our code is available at https://github.com/j-cyoung/GSDatasetDistillation.

Related papers

• Accelerating Large-Scale Dataset Distillation via Exploration-Exploitation Optimization [20.112935166158135]
  We propose Exploration-Exploitation Distillation (E$2$D), a simple, practical method that minimizes redundant computation.<n>We evaluate E$2$D on large-scale benchmarks, surpassing the state-of-the-art on ImageNet-1K while being $18times$ faster.<n>These results demonstrate that targeted, redundancy-reducing updates, rather than brute-force optimization, bridge the gap between accuracy and efficiency in large-scale dataset distillation.
  arXiv  Detail & Related papers  (2026-02-17T00:27:58Z)
• ImLoc: Revisiting Visual Localization with Image-based Representation [61.282162006394934]
  We propose to augment each image with estimated depth maps to capture the geometric structure.<n>This representation is easy to build and maintain, but achieves highest accuracy in challenging conditions.<n>Our method achieves a new state-of-the-art accuracy on various standard benchmarks and outperforms existing memory-efficient methods at comparable map sizes.
  arXiv  Detail & Related papers  (2026-01-07T18:51:51Z)
• Contour Information Aware 2D Gaussian Splatting for Image Representation [0.0]
  We propose a Contour Information-Aware 2D Gaussian Splatting framework.<n>Our method achieves higher reconstruction quality around object edges compared to existing 2DGS methods.
  arXiv  Detail & Related papers  (2025-12-29T07:24:36Z)
• Vision-Language Alignment from Compressed Image Representations using 2D Gaussian Splatting [4.2390854432099205]
  Modern vision language pipelines are driven by RGB vision encoders trained on massive image text corpora.<n>These pipelines inherit two structural inefficiencies from the pixel domain: (i) transmitting dense RGB images from edge devices to the cloud is energy intensive and costly, and (ii) patch based tokenization explodes sequence length.<n>We explore 2D Gaussian Splatting (2DGS) as an alternative visual substrate for alignment.
  arXiv  Detail & Related papers  (2025-09-26T17:41:57Z)
• Emphasizing Discriminative Features for Dataset Distillation in Complex Scenarios [60.470289963986716]
  We propose EDF, a dataset distillation method that enhances key discriminative regions in synthetic images.<n>Our approach is inspired by a key observation: in simple datasets, high-activation areas occupy most of the image, whereas in complex scenarios, the size of these areas is much smaller.<n>In particular, EDF consistently outperforms SOTA results in complex scenarios, such as ImageNet-1K subsets.
  arXiv  Detail & Related papers  (2024-10-22T17:13:19Z)
• Image-GS: Content-Adaptive Image Representation via 2D Gaussians [52.598772767324036]
  We introduce Image-GS, a content-adaptive image representation based on 2D Gaussians radiance.<n>It supports hardware-friendly rapid access for real-time usage, requiring only 0.3K MACs to decode a pixel.<n>We demonstrate its versatility with several applications, including texture compression, semantics-aware compression, and joint image compression and restoration.
  arXiv  Detail & Related papers  (2024-07-02T00:45:21Z)
• Curriculum Dataset Distillation [33.167484258219766]
  We present a curriculum-based dataset distillation framework aiming to harmonize performance and scalability.<n>This framework strategically distills synthetic images, adhering to a curriculum that transitions from simple to complex.<n>Our framework sets new benchmarks in large-scale dataset distillation, achieving substantial improvements of 11.1% on Tiny-ImageNet, 9.0% on ImageNet-1K, and 7.3% on ImageNet-21K.
  arXiv  Detail & Related papers  (2024-05-15T07:27:14Z)
• CompGS: Efficient 3D Scene Representation via Compressed Gaussian Splatting [68.94594215660473]
  We propose an efficient 3D scene representation, named Compressed Gaussian Splatting (CompGS) We exploit a small set of anchor primitives for prediction, allowing the majority of primitives to be encapsulated into highly compact residual forms. Experimental results show that the proposed CompGS significantly outperforms existing methods, achieving superior compactness in 3D scene representation without compromising model accuracy and rendering quality.
  arXiv  Detail & Related papers  (2024-04-15T04:50:39Z)
• Leveraging Representations from Intermediate Encoder-blocks for Synthetic Image Detection [13.840950434728533]
  State-of-the-art Synthetic Image Detection (SID) research has led to strong evidence on the advantages of feature extraction from foundation models. We leverage the image representations extracted by intermediate Transformer blocks of CLIP's image-encoder via a lightweight network. Our method is compared against the state-of-the-art by evaluating it on 20 test datasets and exhibits an average +10.6% absolute performance improvement.
  arXiv  Detail & Related papers  (2024-02-29T12:18:43Z)
• Dataset Quantization [72.61936019738076]
  We present dataset quantization (DQ), a new framework to compress large-scale datasets into small subsets. DQ is the first method that can successfully distill large-scale datasets such as ImageNet-1k with a state-of-the-art compression ratio.
  arXiv  Detail & Related papers  (2023-08-21T07:24:29Z)
• Closed-Loop Transcription via Convolutional Sparse Coding [29.75613581643052]
  Autoencoders often use generic deep networks as the encoder or decoder, which are difficult to interpret. In this work, we make the explicit assumption that the image distribution is generated from a multistage convolution sparse coding (CSC) Our method enjoys several side benefits, including more structured and interpretable representations, more stable convergence, and scalability to large datasets.
  arXiv  Detail & Related papers  (2023-02-18T14:40:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.