Reliable and Resilient Collective Communication Library for LLM Training and Serving

• URL: http://arxiv.org/abs/2512.25059v1
• Date: Wed, 31 Dec 2025 18:53:11 GMT
• Title: Reliable and Resilient Collective Communication Library for LLM Training and Serving
• Abstract summary: We present R$2$CCL, a fault-tolerant communication library for multi-NIC hardware.<n>R$2$CCL performs rapid connection migration, bandwidth-aware load redistribution, and resilient collective algorithms to maintain progress under failures.<n>Experiments show that R$2$CCL is highly robust to NIC failures, incurring less than 1% training and less than 3% inference overheads.
• Score: 5.400838203617311
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Modern ML training and inference now span tens to tens of thousands of GPUs, where network faults can waste 10--15\% of GPU hours due to slow recovery. Common network errors and link fluctuations trigger timeouts that often terminate entire jobs, forcing expensive checkpoint rollback during training and request reprocessing during inference. We present R$^2$CCL, a fault-tolerant communication library that provides lossless, low-overhead failover by exploiting multi-NIC hardware. R$^2$CCL performs rapid connection migration, bandwidth-aware load redistribution, and resilient collective algorithms to maintain progress under failures. We evaluate R$^2$CCL on two 8-GPU H100 InfiniBand servers and via large-scale ML simulators modeling hundreds of GPUs with diverse failure patterns. Experiments show that R$^2$CCL is highly robust to NIC failures, incurring less than 1\% training and less than 3\% inference overheads. R$^2$CCL outperforms baselines AdapCC and DejaVu by 12.18$\times$ and 47$\times$, respectively.

Related papers

• ReCoVer: Resilient LLM Pre-Training System via Fault-Tolerant Collective and Versatile Workload [18.933354857797056]
  Pre-training large language models on massive GPU clusters has made hardware faults routine rather than rare.<n>We propose ReCoVer, a resilient pre-training system that upholds a single invariant.<n>For comparison with checkpoint-and-restart baselines, ReCoVer demonstrates $2.23times$ higher effective throughput after successive failures.
  arXiv  Detail & Related papers  (2026-05-11T20:28:31Z)
• Resource-Efficient Iterative LLM-Based NAS with Feedback Memory [49.44875022114861]
  Neural Architecture Search (NAS) automates network design, but conventional methods demand substantial computational resources.<n>We propose a closed-loop pipeline leveraging large language models (LLMs) to iteratively generate, evaluate, and refine convolutional neural network architectures.
  arXiv  Detail & Related papers  (2026-03-12T16:00:22Z)
• Training LLMs with Fault Tolerant HSDP on 100,000 GPUs [9.97532556913539]
  synchronous training results in a low efficiency due to frequent failures and long recovery time.<n>We propose Fault Tolerant Hybrid-Shared Data Parallelism (FT-HSDP)<n>FT-HSDP uses data parallel replicas as units of fault tolerance.
  arXiv  Detail & Related papers  (2026-01-30T19:57:36Z)
• RollArt: Scaling Agentic RL Training via Disaggregated Infrastructure [49.88201789074532]
  Agentic Reinforcement Learning (RL) enables Large Language Models (LLMs) to perform autonomous decision-making and long-term planning.<n>We present RollArc, a distributed system designed to maximize throughput for multi-task agentic RL on disaggregated infrastructure.
  arXiv  Detail & Related papers  (2025-12-27T11:14:23Z)
• CycleSL: Server-Client Cyclical Update Driven Scalable Split Learning [60.59553507555341]
  We introduce CycleSL, a novel aggregation-free split learning framework.<n>Inspired by alternating block coordinate descent, CycleSL treats server-side training as an independent higher-level machine learning task.<n>Our empirical findings highlight the effectiveness of CycleSL in enhancing model performance.
  arXiv  Detail & Related papers  (2025-11-23T21:00:21Z)
• DataStates-LLM: Lazy Asynchronous Checkpointing for Large Language Models [3.3484462092188005]
  We introduce a lazy asynchronous multi-level approach that takes advantage of the fact that the tensors making up the model and state shards remain immutable for extended periods of time. The results show up to 48$times$ faster checkpointing and 2.2$times$ faster end-to-end training compared with state-of-art checkpointing approaches.
  arXiv  Detail & Related papers  (2024-06-15T18:30:40Z)
• Hardware-Aware Parallel Prompt Decoding for Memory-Efficient Acceleration of LLM Inference [23.633481089469836]
  Auto-regressive decoding of Large Language Models (LLMs) results in significant overheads in their hardware performance.<n>We propose a novel parallel prompt decoding that requires only $0.0002$% trainable parameters, enabling efficient training on a single A100-40GB GPU in just 16 hours.<n>Our approach demonstrates up to 2.49$times$ speedup and maintains a minimal memory overhead of just $0.0004$%.
  arXiv  Detail & Related papers  (2024-05-28T22:19:30Z)
• ReCycle: Resilient Training of Large DNNs using Pipeline Adaptation [2.0181279529015925]
  ReCycle is a system designed for efficient training in the presence of failures. It exploits the inherent functional redundancy in distributed training systems. We show it achieves high training throughput under multiple failures.
  arXiv  Detail & Related papers  (2024-05-22T21:35:56Z)
• A Quadratic Synchronization Rule for Distributed Deep Learning [66.68264684667562]
  This work proposes a theory-grounded method for determining $H$, named the Quadratic Synchronization Rule (QSR) Experiments on ResNet and ViT show that local gradient methods with QSR consistently improve the test accuracy over other synchronization strategies.
  arXiv  Detail & Related papers  (2023-10-22T21:38:57Z)
• Implementing Reinforcement Learning Datacenter Congestion Control in NVIDIA NICs [64.26714148634228]
  congestion control (CC) algorithms become extremely difficult to design. It is currently not possible to deploy AI models on network devices due to their limited computational capabilities. We build a computationally-light solution based on a recent reinforcement learning CC algorithm.
  arXiv  Detail & Related papers  (2022-07-05T20:42:24Z)
• Aryl: An Elastic Cluster Scheduler for Deep Learning [12.942546041713596]
  We introduce Aryl, a new cluster scheduler to address problems for both training and inference. Aryl introduces capacity loaning to loan idle inference servers for training jobs. It improves cluster usage by up to 26.9% over the cluster scheduler without capacity loaning or elastic scaling.
  arXiv  Detail & Related papers  (2022-02-16T07:03:25Z)
• Accelerating Training and Inference of Graph Neural Networks with Fast Sampling and Pipelining [58.10436813430554]
  Mini-batch training of graph neural networks (GNNs) requires a lot of computation and data movement. We argue in favor of performing mini-batch training with neighborhood sampling in a distributed multi-GPU environment. We present a sequence of improvements to mitigate these bottlenecks, including a performance-engineered neighborhood sampler. We also conduct an empirical analysis that supports the use of sampling for inference, showing that test accuracies are not materially compromised.
  arXiv  Detail & Related papers  (2021-10-16T02:41:35Z)
• ProgFed: Effective, Communication, and Computation Efficient Federated Learning by Progressive Training [65.68511423300812]
  We propose ProgFed, a progressive training framework for efficient and effective federated learning. ProgFed inherently reduces computation and two-way communication costs while maintaining the strong performance of the final models. Our results show that ProgFed converges at the same rate as standard training on full models.
  arXiv  Detail & Related papers  (2021-10-11T14:45:00Z)
• Highly Available Data Parallel ML training on Mesh Networks [0.34265828682659694]
  On a mesh network this is challenging because failures will create holes in the mesh. We present techniques to route all traffic around failed chips on 2-D meshes.
  arXiv  Detail & Related papers  (2020-11-06T21:36:16Z)
• Communication Contention Aware Scheduling of Multiple Deep Learning Training Jobs [17.45154289084637]
  We establish a new DDL job scheduling framework which organizes DDL jobs as Directed Acyclic Graphs (DAGs) We then propose an efficient algorithm, LWF-$kappa$, to balance the GPU utilization and consolidate the allocated GPU for each job. We show that LWF-$kappa$ achieves up to $1.59times$ improvement over the classical first-fit algorithms.
  arXiv  Detail & Related papers  (2020-02-24T07:50:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.