PubSwap: Public-Data Off-Policy Coordination for Federated RLVR

• URL: http://arxiv.org/abs/2604.12160v1
• Date: Tue, 14 Apr 2026 00:35:39 GMT
• Title: PubSwap: Public-Data Off-Policy Coordination for Federated RLVR
• Abstract summary: We propose a federated RLVR framework that combines LoRA-based local adaptation with public-data-based off-policy steps to improve both communication efficiency and cross-client coordination.<n>Our results highlight a simple and effective recipe for federated reasoning post-training: combining low-rank communication with limited public-data coordination.
• Score: 26.391752418306677
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reasoning post-training with reinforcement learning from verifiable rewards (RLVR) is typically studied in centralized settings, yet many realistic applications involve decentralized private data distributed across organizations. Federated training is a natural solution, but scaling RLVR in this regime is challenging: full-model synchronization is expensive, and performing many local steps can cause severe client drift under heterogeneous data. We propose a federated RLVR framework that combines LoRA-based local adaptation with public-data-based off-policy steps to improve both communication efficiency and cross-client coordination. In particular, a small shared public dataset is used to periodically exchange and reuse response-level training signals across organizations, providing a lightweight anchor toward a more globally aligned objective without exposing private data. Our method selectively replaces locally incorrect responses with globally correct ones during public-data steps, thereby keeping training closer to the local policy while still benefiting from cross-client coordination. Across mathematical and medical reasoning benchmarks and models, our method consistently improves over standard baselines. Our results highlight a simple and effective recipe for federated reasoning post-training: combining low-rank communication with limited public-data coordination.

Related papers

• Personalized Federated Learning with Bidirectional Communication Compression via One-Bit Random Sketching [19.959825317183924]
  Federated Learning (FL) enables collaborative training across decentralized data.<n>We propose pFed1BS, a novel personalized FL framework that achieves extreme communication compression through one-bit random sketching.<n>In our framework, clients transmit highly compressed one-bit sketches, and the server aggregates and broadcasts a global one-bit consensus.
  arXiv  Detail & Related papers  (2025-11-17T08:55:22Z)
• Accelerating Privacy-Preserving Federated Learning in Large-Scale LEO Satellite Systems [57.692181589325116]
  Large-scale low-Earth-orbit (LEO) satellite systems are increasingly valued for their ability to enable rapid and wide-area data exchange.<n>Due to privacy concerns and regulatory constraints, raw data collected at remote clients cannot be centrally aggregated.<n>Federated learning offers a privacy-preserving alternative by training local models on distributed devices and exchanging only model parameters.<n>We propose a discrete temporal graph-based on-demand scheduling framework that dynamically allocates communication resources to accelerate federated learning.
  arXiv  Detail & Related papers  (2025-09-05T03:33:42Z)
• FedComLoc: Communication-Efficient Distributed Training of Sparse and Quantized Models [52.13056951033747]
  Federated Learning (FL) has garnered increasing attention due to its unique characteristic of allowing heterogeneous clients to process their private data locally and interact with a central server.<n>A critical bottleneck in FL is the communication cost.<n>Our work is inspired by the innovative Scaffnew algorithm, which has advanced the reduction of communication complexity in FL.<n>We introduce FedComLoc, integrating practical and effective compression into Scaffnew to further enhance communication efficiency.
  arXiv  Detail & Related papers  (2024-03-14T22:29:59Z)
• FedLPA: One-shot Federated Learning with Layer-Wise Posterior Aggregation [7.052566906745796]
  FedLPA is a layer-wise posterior aggregation method for federated learning. We show that FedLPA significantly improves learning performance over state-of-the-art methods across several metrics.
  arXiv  Detail & Related papers  (2023-09-30T10:51:27Z)
• FedAgg: Adaptive Federated Learning with Aggregated Gradients [1.5653612447564105]
  We propose an adaptive FEDerated learning algorithm called FedAgg to alleviate the divergence between the local and average model parameters and obtain a fast model convergence rate. We show that our framework is superior to existing state-of-the-art FL strategies for enhancing model performance and accelerating convergence rate under IID and Non-IID datasets.
  arXiv  Detail & Related papers  (2023-03-28T08:07:28Z)
• GradSkip: Communication-Accelerated Local Gradient Methods with Better Computational Complexity [54.585248253601314]
  We study a class of distributed optimization algorithms that aim to alleviate high communication costs by allowing clients to perform multiple local gradient-type training steps before communication.<n>In particular, we prove that our modified method, GradSkip, converges linearly under the same assumptions and has the same accelerated communication complexity.
  arXiv  Detail & Related papers  (2022-10-28T20:59:06Z)
• DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Training [84.81043932706375]
  We propose a novel personalized federated learning framework in a decentralized (peer-to-peer) communication protocol named Dis-PFL. Dis-PFL employs personalized sparse masks to customize sparse local models on the edge. We demonstrate that our method can easily adapt to heterogeneous local clients with varying computation complexities.
  arXiv  Detail & Related papers  (2022-06-01T02:20:57Z)
• Federated Learning from Small Datasets [48.879172201462445]
  Federated learning allows multiple parties to collaboratively train a joint model without sharing local data. We propose a novel approach that intertwines model aggregations with permutations of local models. The permutations expose each local model to a daisy chain of local datasets resulting in more efficient training in data-sparse domains.
  arXiv  Detail & Related papers  (2021-10-07T13:49:23Z)
• Scalable Multi-Agent Reinforcement Learning for Residential Load Scheduling under Data Governance [5.37556626581816]
  Multi-agent reinforcement learning (MARL) has made remarkable advances in solving cooperative residential load scheduling problems.<n> centralized training, the most common paradigm for MARL, limits large-scale deployment in communication-constrained cloud-edge environments.<n>Our proposed approach is based on actor-critic methods, where the global critic is a learned function of individual critics computed solely based on local observations of households.
  arXiv  Detail & Related papers  (2021-10-06T14:05:26Z)
• Adaptive Federated Dropout: Improving Communication Efficiency and Generalization for Federated Learning [6.982736900950362]
  A revolutionary decentralized machine learning setting, known as Federated Learning, enables multiple clients located at different geographical locations to collaboratively learn a machine learning model. Communication between the clients and the server is considered a main bottleneck in the convergence time of federated learning. We propose and study Adaptive Federated Dropout (AFD), a novel technique to reduce the communication costs associated with federated learning.
  arXiv  Detail & Related papers  (2020-11-08T18:41:44Z)
• Decentralised Learning from Independent Multi-Domain Labels for Person Re-Identification [69.29602103582782]
  Deep learning has been successful for many computer vision tasks due to the availability of shared and centralised large-scale training data. However, increasing awareness of privacy concerns poses new challenges to deep learning, especially for person re-identification (Re-ID) We propose a novel paradigm called Federated Person Re-Identification (FedReID) to construct a generalisable global model (a central server) by simultaneously learning with multiple privacy-preserved local models (local clients) This client-server collaborative learning process is iteratively performed under privacy control, enabling FedReID to realise decentralised learning without sharing distributed data nor collecting any
  arXiv  Detail & Related papers  (2020-06-07T13:32:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.