UCCL-Zip: Lossless Compression Supercharged GPU Communication

• URL: http://arxiv.org/abs/2604.17172v2
• Date: Tue, 21 Apr 2026 04:46:04 GMT
• Title: UCCL-Zip: Lossless Compression Supercharged GPU Communication
• Abstract summary: UCCL-Zip supports both point-to-point (P2P) and collective communication without modifying user-facing APIs or compromising numerical correctness.<n>Uzip-P2P employs a split-send pipeline that exposes transmissible data early and overlaps compression with communication.<n>Uzip-NCCL integrates compression into NCCL's persistent kernel model via fused execution.
• Score: 37.95759514379356
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The rapid growth of large language models (LLMs) has made GPU communication a critical bottleneck. While prior work reduces communication volume via quantization or lossy compression, these approaches introduce numerical errors that can degrade convergence, accuracy, and stability. We present UCCL-Zip, a unified design that integrates lossless compression directly into GPU communication primitives. UCCL-Zip supports both point-to-point (P2P) and collective communication without modifying user-facing APIs or compromising numerical correctness. For P2P communication, Uzip-P2P employs a split-send pipeline that exposes transmissible data early and overlaps compression with communication, while preserving high GPU efficiency by operating on large data blocks. For collective communication, Uzip-NCCL integrates compression into NCCL's persistent kernel model via fused execution, eliminating redundant memory traffic and kernel launches. In real workloads, UCCL-Zip accelerates RL weight synchronization by up to 47.5% and reduces vLLM end-to-end inference latency by up to 10%, all without application changes.

Related papers

• Communication-Efficient LLM Adaptation over Decentralized GPU Meshes [48.95736135870457]
  We propose an asynchronous two-circuit system for post-pretraining adaptation.<n>A fast compressed training circuit drives throughput using masking activation for pipeline-parallel transfer and compressed data-parallel synchronization.<n>We show that masking supports post-pretraining adaptation at high compression rates when anchored this way.
  arXiv  Detail & Related papers  (2026-09-13T06:56:50Z)
• Profiling-Driven Adaptive Distributed Transformer Inference on Embedded Edge Deployment [60.442064966340524]
  We present a prototype study on NVIDIA Jetson Orin Nano devices connected over WiFi.<n>Key finding is that the dominant bottleneck is not just network bandwidth but also the CPU- GPU staging during communication.<n>Experiments show that this strategy reduces latency by 65%-77% and energy consumption by 34%-52% relative to full-tensor exchange.
  arXiv  Detail & Related papers  (2026-05-25T10:39:28Z)
• ZipCCL: Efficient Lossless Data Compression of Communication Collectives for Accelerating LLM Training [21.144281084668346]
  Communication has emerged as a critical bottleneck in the distributed training of large language models.<n>We introduce ZipCCL, a compressed communication library of collectives for training.<n>ZipCCL reduces communication time by up to 1.35$times$ and achieves end-to-end training speedups of up to 1.18$times$ without any impact on model quality.
  arXiv  Detail & Related papers  (2026-04-30T13:29:59Z)
• ZipServ: Fast and Memory-Efficient LLM Inference with Hardware-Aware Lossless Compression [19.538318240352424]
  Lossless model compression holds tremendous promise for alleviating the memory and bandwidth bottlenecks in bit-exact Large Language Model (LLM) serving.<n>Existing approaches often result in substantial inference slowdowns due to fundamental design mismatches with GPU architectures.<n>We present ZipServ, a lossless compression framework co-designed for efficient LLM inference.
  arXiv  Detail & Related papers  (2026-03-18T07:21:21Z)
• NCCL EP: Towards a Unified Expert Parallel Communication API for NCCL [5.243473994390624]
  NCCL EP (Expert Parallelism) is a ground-up MoE communication library built entirely on NCCL's Device API.<n>It provides unified ncclEpDispatch and ncclEpCombine primitives with both C and Python interfaces.<n>We evaluate NCCL EP on an H100-based cluster across multi-node configurations, demonstrating competitive LL kernel performance and presenting end-to-end results with vLLM integration.
  arXiv  Detail & Related papers  (2026-03-13T21:28:22Z)
• HetCCL: Accelerating LLM Training with Heterogeneous GPUs [5.820335672510985]
  HetCCL is a collective communication library that unifies vendor-specific backends and enables RDMA-based communication across GPUs without requiring driver modifications.<n>HetCCL matches NCCL and RCCL performance in homogeneous setups while uniquely scaling in heterogeneous environments.
  arXiv  Detail & Related papers  (2026-01-30T05:31:29Z)
• Lossless Compression for LLM Tensor Incremental Snapshots [0.0]
  We build an effective compression solution, known as Language Model (LMC)<n>We show that a 16-core parallel implementation of LMC can attain compression and decompression throughput of 2.78 GiB/s and 3.76 GiB/s respectively.<n>This increase in performance ultimately reduces the resources needed and provides more time to copy the data to the storage system before the next epoch thus allowing for higher-frequency checkpoints.
  arXiv  Detail & Related papers  (2025-05-14T21:24:14Z)
• The Big Send-off: High Performance Collectives on GPU-based Supercomputers [1.852182371724987]
  Existing libraries such as RCCL and Cray-MPICH exhibit critical limitations on systems such as Frontier.<n>We introduce PCCL, a communication library with highly optimized implementations of all-gather and reduce-scatter operations.<n>It achieves substantial performance improvements, delivering 6-33x speedups over RCCL and 28-70x over Cray-MPICH for all-gather on 2048 GCDs of Frontier.
  arXiv  Detail & Related papers  (2025-04-25T19:23:46Z)
• EoRA: Fine-tuning-free Compensation for Compressed LLM with Eigenspace Low-Rank Approximation [84.70637613266835]
  EoRA is a fine-tuning-free method that augments compressed Large Language Models with low-rank matrices.<n>EoRA consistently outperforms prior training-free low rank methods in recovering the accuracy of compressed LLMs.
  arXiv  Detail & Related papers  (2024-10-28T17:59:03Z)
• Accelerating Large Language Model Training with Hybrid GPU-based Compression [3.204387803072905]
  MPI libraries have been proven to reduce message size significantly and leverage interconnect bandwidth. We investigate the efficacy of compression-assisted MPI collectives under the context of distributed Large Language Model (LLM) training.
  arXiv  Detail & Related papers  (2024-09-04T04:05:30Z)
• FDC: Fast KV Dimensionality Compression for Efficient LLM Inference [11.194752361478567]
  FDC is a fast KV dimensionality compression system that eliminates the decompression overhead incurred in the existing KV dimensionality compression system, Palu, and reduces attention time.<n>In experiments, FDC can reduce Job Completion Time (JCT) by up to 64%, and delivers up to 1.97X throughput under the same latency.<n>When state-of-the-art eviction and quantization methods are combined with FDC, they exhibit similar improvements compared to those combined with Palu.
  arXiv  Detail & Related papers  (2024-08-07T22:10:26Z)
• LoCo: Low-Bit Communication Adaptor for Large-scale Model Training [63.040522637816906]
  Low-bit communication often degrades training quality due to compression information loss.<n>We propose Low-bit Communication Adaptor (LoCo), which compensates local local GPU nodes before, without compromising quality.<n> Experimental results show that across moving large-scale training model frameworks like Megatron-LM and PyTorchs FSDP, LoCo significantly improves compression communication efficiency.
  arXiv  Detail & Related papers  (2024-07-05T13:01:36Z)
• LoCoDL: Communication-Efficient Distributed Learning with Local Training and Compression [56.01900711954956]
  We introduce LoCoDL, a communication-efficient algorithm that leverages the two popular and effective techniques of Local training, which reduces the communication frequency, and Compression, in which short bitstreams are sent instead of full-dimensional vectors of floats.<n>LoCoDL provably benefits from local training and compression and enjoys a doubly-accelerated communication complexity, with respect to the condition number of the functions and the model dimension, in the general heterogenous regime with strongly convex functions.
  arXiv  Detail & Related papers  (2024-03-07T09:22:50Z)
• Fed-CVLC: Compressing Federated Learning Communications with Variable-Length Codes [54.18186259484828]
  In Federated Learning (FL) paradigm, a parameter server (PS) concurrently communicates with distributed participating clients for model collection, update aggregation, and model distribution over multiple rounds. We show strong evidences that variable-length is beneficial for compression in FL. We present Fed-CVLC (Federated Learning Compression with Variable-Length Codes), which fine-tunes the code length in response to the dynamics of model updates.
  arXiv  Detail & Related papers  (2024-02-06T07:25:21Z)
• Efficient Adversarial Contrastive Learning via Robustness-Aware Coreset Selection [59.77647907277523]
  Adversarial contrast learning (ACL) does not require expensive data annotations but outputs a robust representation that withstands adversarial attacks. ACL needs tremendous running time to generate the adversarial variants of all training data. This paper proposes a robustness-aware coreset selection (RCS) method to speed up ACL.
  arXiv  Detail & Related papers  (2023-02-08T03:20:14Z)
• Synthesizing Collective Communication Algorithms for Heterogeneous Networks with TACCL [1.5528708400965123]
  We present TACCL, a synthesizer for collective communication primitives for large-scale multi-GPU systems. TACCL encodes a profiled topology and input size into a synthesis problem to generate optimized communication algorithms. Using TACCL's algorithms speeds up the end-to-end training of an internal mixture of experts model by $17%$.
  arXiv  Detail & Related papers  (2021-11-08T23:20:52Z)
• Towards Compact CNNs via Collaborative Compression [166.86915086497433]
  We propose a Collaborative Compression scheme, which joints channel pruning and tensor decomposition to compress CNN models. We achieve 52.9% FLOPs reduction by removing 48.4% parameters on ResNet-50 with only a Top-1 accuracy drop of 0.56% on ImageNet 2012.
  arXiv  Detail & Related papers  (2021-05-24T12:07:38Z)
• 1-bit LAMB: Communication Efficient Large-Scale Large-Batch Training with LAMB's Convergence Speed [17.953619054149378]
  We propose a new communication-efficient algorithm, 1-bit LAMB, which supports adaptive layerwise learning rates even when communication is compressed. For BERT-Large pre-training task with batch sizes from 8K to 64K, our evaluations demonstrate that 1-bit LAMB with NCCL-based backend is able to achieve up to 4.6x communication volume reduction.
  arXiv  Detail & Related papers  (2021-04-13T10:07:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.