dreaMLearning: Data Compression Assisted Machine Learning

• URL: http://arxiv.org/abs/2506.22190v1
• Date: Fri, 27 Jun 2025 12:57:22 GMT
• Title: dreaMLearning: Data Compression Assisted Machine Learning
• Authors: Xiaobo Zhao, Aaron Hurst, Panagiotis Karras, Daniel E. Lucani,
• Abstract summary: This paper introduces dreaMLearning, a framework that enables learning from compressed data without decompression.<n>Experiments demonstrate that dreaMLearning accelerates training by up to 8.8x, reduces memory usage by 10x, and cuts storage by 42%.<n>These advancements enhance diverse ML applications, including distributed and federated learning, and tinyML on resource-constrained edge devices.
• Score: 12.875773186735396
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite rapid advancements, machine learning, particularly deep learning, is hindered by the need for large amounts of labeled data to learn meaningful patterns without overfitting and immense demands for computation and storage, which motivate research into architectures that can achieve good performance with fewer resources. This paper introduces dreaMLearning, a novel framework that enables learning from compressed data without decompression, built upon Entropy-based Generalized Deduplication (EntroGeDe), an entropy-driven lossless compression method that consolidates information into a compact set of representative samples. DreaMLearning accommodates a wide range of data types, tasks, and model architectures. Extensive experiments on regression and classification tasks with tabular and image data demonstrate that dreaMLearning accelerates training by up to 8.8x, reduces memory usage by 10x, and cuts storage by 42%, with a minimal impact on model performance. These advancements enhance diverse ML applications, including distributed and federated learning, and tinyML on resource-constrained edge devices, unlocking new possibilities for efficient and scalable learning.

Related papers

• SPaRFT: Self-Paced Reinforcement Fine-Tuning for Large Language Models [51.74498855100541]
  Large language models (LLMs) have shown strong reasoning capabilities when fine-tuned with reinforcement learning (RL)<n>We propose textbfSPaRFT, a self-paced learning framework that enables efficient learning based on the capability of the model being trained.
  arXiv  Detail & Related papers  (2025-08-07T03:50:48Z)
• Investigating Structural Pruning and Recovery Techniques for Compressing Multimodal Large Language Models: An Empirical Study [64.26593350748401]
  Multimodal Large Language Models (MLLMs) demonstrate impressive capabilities.<n>Current parameter reduction techniques primarily involve training MLLMs from Small Language Models (SLMs)<n>We propose to directly compress existing MLLMs through structural pruning combined with efficient recovery training.
  arXiv  Detail & Related papers  (2025-07-28T11:57:52Z)
• MiniCPM4: Ultra-Efficient LLMs on End Devices [124.73631357883228]
  MiniCPM4 is a highly efficient large language model (LLM) designed explicitly for end-side devices.<n>We achieve this efficiency through systematic innovation in four key dimensions: model architecture, training data, training algorithms, and inference systems.<n>MiniCPM4 is available in two versions, with 0.5B and 8B parameters, respectively.
  arXiv  Detail & Related papers  (2025-06-09T16:16:50Z)
• Flyweight FLIM Networks for Salient Object Detection in Biomedical Images [42.763966145188625]
  This study presents methods to learn dilated-separable convolutional kernels and multi-dilation layers without backpropagation for FLIM networks.<n>It also proposes a novel network simplification method to reduce kernel redundancy and encoder size.
  arXiv  Detail & Related papers  (2025-04-15T11:57:40Z)
• Leveraging Large Language Models to Address Data Scarcity in Machine Learning: Applications in Graphene Synthesis [0.0]
  Machine learning in materials science faces challenges due to limited experimental data.<n>We propose strategies that utilize large language models (LLMs) to enhance machine learning performance.
  arXiv  Detail & Related papers  (2025-03-06T16:04:01Z)
• Learning with Less: Knowledge Distillation from Large Language Models via Unlabeled Data [54.934578742209716]
  In real-world NLP applications, Large Language Models (LLMs) offer promising solutions due to their extensive training on vast datasets.<n>LLKD is an adaptive sample selection method that incorporates signals from both the teacher and student.<n>Our comprehensive experiments show that LLKD achieves superior performance across various datasets with higher data efficiency.
  arXiv  Detail & Related papers  (2024-11-12T18:57:59Z)
• Bullion: A Column Store for Machine Learning [4.096087402737292]
  This paper presents Bullion, a columnar storage system tailored for machine learning workloads. Bundy addresses the complexities of data compliance, optimize the encoding of long sequence sparse features, efficiently manages wide-table projections, introduces feature quantization in storage, and provides a comprehensive cascading encoding framework. Preliminary experimental results and theoretical analysis demonstrate Bullion's improved ability to deliver strong performance in the face of the unique demands of machine learning workloads.
  arXiv  Detail & Related papers  (2024-04-13T05:01:54Z)
• Retrieval-based Knowledge Transfer: An Effective Approach for Extreme Large Language Model Compression [64.07696663255155]
  Large-scale pre-trained language models (LLMs) have demonstrated exceptional performance in various natural language processing (NLP) tasks. However, the massive size of these models poses huge challenges for their deployment in real-world applications. We introduce a novel compression paradigm called Retrieval-based Knowledge Transfer (RetriKT) which effectively transfers the knowledge of LLMs to extremely small-scale models.
  arXiv  Detail & Related papers  (2023-10-24T07:58:20Z)
• Dataset Quantization [72.61936019738076]
  We present dataset quantization (DQ), a new framework to compress large-scale datasets into small subsets. DQ is the first method that can successfully distill large-scale datasets such as ImageNet-1k with a state-of-the-art compression ratio.
  arXiv  Detail & Related papers  (2023-08-21T07:24:29Z)
• IB-DRR: Incremental Learning with Information-Back Discrete Representation Replay [4.8666876477091865]
  Incremental learning aims to enable machine learning models to continuously acquire new knowledge given new classes. Saving a subset of training samples of previously seen classes in the memory and replaying them during new training phases is proven to be an efficient and effective way to fulfil this aim. However, finding a trade-off between the model performance and the number of samples to save for each class is still an open problem for replay-based incremental learning.
  arXiv  Detail & Related papers  (2021-04-21T15:32:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.