Efficient Learned Data Compression via Dual-Stream Feature Decoupling

• URL: http://arxiv.org/abs/2604.07239v1
• Date: Wed, 08 Apr 2026 16:05:24 GMT
• Title: Efficient Learned Data Compression via Dual-Stream Feature Decoupling
• Abstract summary: We propose a Dual-Stream Multi-Scale Decoupler that disentangles local and global contexts to replace deep serial processing with shallow parallel streams.<n>Our method achieves state-of-the-art performance in both compression ratio and throughput, while maintaining the lowest latency and memory usage.
• Score: 11.752785837648199
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While Learned Data Compression (LDC) has achieved superior compression ratios, balancing precise probability modeling with system efficiency remains challenging. Crucially, uniform single-stream architectures struggle to simultaneously capture micro-syntactic and macro-semantic features, necessitating deep serial stacking that exacerbates latency. Compounding this, heterogeneous systems are constrained by device speed mismatches, where throughput is capped by Amdahl's Law due to serial processing. To this end, we propose a Dual-Stream Multi-Scale Decoupler that disentangles local and global contexts to replace deep serial processing with shallow parallel streams, and incorporate a Hierarchical Gated Refiner for adaptive feature refinement and precise probability modeling. Furthermore, we design a Concurrent Stream-Parallel Pipeline, which overcomes systemic bottlenecks to achieve full-pipeline parallelism. Extensive experiments demonstrate that our method achieves state-of-the-art performance in both compression ratio and throughput, while maintaining the lowest latency and memory usage. The code is available at https://github.com/huidong-ma/FADE.

Related papers

• Communication-Efficient LLM Adaptation over Decentralized GPU Meshes [48.95736135870457]
  We propose an asynchronous two-circuit system for post-pretraining adaptation.<n>A fast compressed training circuit drives throughput using masking activation for pipeline-parallel transfer and compressed data-parallel synchronization.<n>We show that masking supports post-pretraining adaptation at high compression rates when anchored this way.
  arXiv  Detail & Related papers  (2026-09-13T06:56:50Z)
• FLARE: Diffusion for Hybrid Language Model [72.60770374799634]
  FLARE is a systematic conversion framework for hybrid-attention large language models.<n>It combines a token-equal AR-and-diffusion objective, hardware-aware kernels, and unified inference, enabling one checkpoint to support both AR-style verified decoding and diffusion-style parallel denoising.<n>Our results suggest practical dLLMs are limited not only by decoding algorithms, but also by transfer data quality and the training inefficiency of current block-diffusion objectives.
  arXiv  Detail & Related papers  (2026-06-01T06:58:15Z)
• ResBM: Residual Bottleneck Models for Low-Bandwidth Pipeline Parallelism [0.0]
  Large-scale decentralized training can utilize otherwise untapped compute resources.<n>Recent efforts have claimed up to 100x activation compression but rely on complex constrained optimization and diverge from true end-to-end training.<n>We show that ResBMs achieve state-of-the-art 128x activation compression without significant loss in convergence rates and without significant memory or compute overhead.
  arXiv  Detail & Related papers  (2026-04-13T18:40:45Z)
• Towards Practical Lossless Neural Compression for LiDAR Point Clouds [84.36825469211375]
  extreme sparsity of high-precision geometric details hinders efficient context modeling.<n>Our framework comprises two lightweight modules.<n>Experiments demonstrate competitive compression performance at real-time speed.
  arXiv  Detail & Related papers  (2026-03-26T10:02:07Z)
• AdaFuse: Accelerating Dynamic Adapter Inference via Token-Level Pre-Gating and Fused Kernel Optimization [84.25316984309725]
  Integration of dynamic, sparse structures with parameter-efficient adapters (e.g., LoRA) is a powerful technique for enhancing Large Language Models (LLMs)<n>Despite minimal increases in computational load, the computation latency often skyrockets, leading to decoding speeds slowing by over 2.5 times.<n>AdaFuse is a framework built on a tight co-design between the algorithm and the underlying hardware system to enable efficient dynamic adapter execution.
  arXiv  Detail & Related papers  (2026-03-12T12:46:42Z)
• Accelerating Diffusion via Hybrid Data-Pipeline Parallelism Based on Conditional Guidance Scheduling [10.012655130147413]
  Diffusion models have achieved remarkable progress in high-fidelity image, video, and audio generation.<n>Our framework achieves $2.31times$ and $2.07times$ latency reductions on SDXL and SD3, respectively.<n>Our approach also outperforms existing methods in acceleration under high-resolution synthesis settings.
  arXiv  Detail & Related papers  (2026-02-25T10:23:07Z)
• AsyncMesh: Fully Asynchronous Optimization for Data and Pipeline Parallelism [54.8494905524997]
  We introduce asynchronous updates across both parallelism axes, relaxing the co-location requirement.<n>We provide convergence guarantees for both sparse averaging and asynchronous updates.<n>Experiments on large-scale language models demonstrate that our approach matches the performance of the fully synchronous baseline.
  arXiv  Detail & Related papers  (2026-01-30T01:24:47Z)
• DeepCoT: Deep Continual Transformers for Real-Time Inference on Data Streams [63.27233749591346]
  Transformer-based models have dramatically increased their size and parameter count to tackle increasingly complex tasks.<n>Stream data inference is typically performed over a sliding temporal window, leading to highly redundant computations.<n>We propose the Deep Continual Transformer (DeepCoT), a redundancy-free encoder-only model that can be applied over existing deep encoder architectures with minimal changes.
  arXiv  Detail & Related papers  (2025-11-21T16:15:43Z)
• PT$^2$-LLM: Post-Training Ternarization for Large Language Models [52.4629647715623]
  Large Language Models (LLMs) have shown impressive capabilities across diverse tasks, but their large memory and compute demands hinder deployment.<n>We propose PT$2$-LLM, a post-training ternarization framework tailored for LLMs.<n>At its core is an Asymmetric Ternary Quantizer equipped with a two-stage refinement pipeline.
  arXiv  Detail & Related papers  (2025-09-27T03:01:48Z)
• ASPD: Unlocking Adaptive Serial-Parallel Decoding by Exploring Intrinsic Parallelism in LLMs [34.477777651648914]
  Large language models (LLMs) pose significant inference latency challenges due to their autoregressive decoding paradigm.<n>We propose an Adaptive Serial-Parallel Decoding (ASPD) which addresses two core challenges: automated construction of parallelizable data and efficient parallel decoding mechanism.<n>Our framework sets a groundbreaking benchmark for efficient LLM parallel inference, paving the way for its deployment in latency-sensitive applications such as AI-powered customer service bots and answer retrieval engines.
  arXiv  Detail & Related papers  (2025-08-12T12:35:55Z)
• Accelerating Parallel Diffusion Model Serving with Residual Compression [11.093231994205986]
  Diffusion models produce realistic images and videos but require substantial computational resources.<n>Parallel inference introduces significant communication overhead from exchanging large activations between devices.<n>We present CompactFusion, a compression framework that significantly reduces communication while preserving generation quality.
  arXiv  Detail & Related papers  (2025-07-23T13:49:25Z)
• Nesterov Method for Asynchronous Pipeline Parallel Optimization [59.79227116582264]
  We introduce a variant of Nesterov Accelerated Gradient (NAG) for asynchronous optimization in Pipeline Parallelism.<n>Specifically, we modify the look-ahead step in NAG to effectively address the staleness in gradients.<n>We theoretically prove that our approach converges at a sublinear rate in the presence of fixed delay in gradients.
  arXiv  Detail & Related papers  (2025-05-02T08:23:29Z)
• The Streaming Batch Model for Efficient and Fault-Tolerant Heterogeneous Execution [28.768566833298365]
  We present Ray Data, a streaming batch system that improves throughput on heterogeneous batch inference pipelines by 2.5-12$times$.<n>Ray Data improves training throughput for multimodal models such as Stable Diffusion by 31% compared to single-node ML data loaders.
  arXiv  Detail & Related papers  (2025-01-16T19:54:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.