Accurate and Resource-Efficient Federated Continual Learning

• URL: http://arxiv.org/abs/2606.11480v1
• Date: Tue, 09 Jun 2026 22:14:16 GMT
• Title: Accurate and Resource-Efficient Federated Continual Learning
• Abstract summary: Federated continual learning (FCL) must learn from distributed task streams under limited resources.<n>We propose FedRAN, a resource-aware analytic FCL framework that replaces gradient-based updates with compact random feature statistics.<n>FedRAN improves average accuracy by up to 4.8 percentage points over the strongest baseline, uses 30.6-121.8$times$ less per-client communication than optimization-based FCL, and is 190.3$times$ faster on average than gradient-based baselines.
• Score: 3.983545455427237
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated continual learning (FCL) must learn from distributed task streams under limited resources, such as communication, computation, memory, and label availability. Existing FCL methods often rely on repeated local optimization, replay, and full supervision. Analytic alternatives avoid iterative training and replay, but using high-dimensional random features to improve accuracy requires a second-order feature statistic, the Gram matrix, which has a quadratic communication cost in the random feature size $M$. We propose FedRAN, a resource-aware analytic FCL framework that replaces gradient-based updates with compact random feature statistics. Each client transmits a truncated-SVD summary of its Gram matrix, reducing the dominant second-order upload from quadratic to linear in $M$ for fixed rank. The server performs a two-level QR-SVD subspace merge, spatially across clients and temporally across tasks, and solves a ridge classifier in closed form. FedRAN further supports label scarcity through prototype-based pseudo-labeling. Across CIFAR-100, ImageNet-R, and VTAB datasets, FedRAN improves average accuracy by up to 4.8 percentage points over the strongest baseline, uses 30.6-121.8$\times$ less per-client communication than optimization-based FCL, and is 190.3$\times$ faster on average than gradient-based baselines; with only 20% labels, pseudo-labeling improves average accuracy by up to 6.61 points. These results show that FedRAN enables accurate and resource-efficient FCL under communication, computation, and label constraints. The source code is available at https://github.com/JebacyrilArockiaraj/Fed-RAN-SSL.

Related papers

• TabPFN beyond Tabular Data: Calibration and Accuracy on Multimodal Embeddings [2.4483752182160954]
  We evaluate TabPFN as a plug-and-play, zero-gradient classification head for frozen image, text, and audio encoders.<n>Across 22,820 evaluation episodes spanning 14 datasets, 11 encoders, and three modalities, TabPFN achieves the best mean rank among nine classification heads on both negative log-likelihood (NLL) and expected calibration error (ECE)<n>Its accuracy advantage is conditional, concentrating at moderate-to-high shot counts and low-to-moderate feature dimensions.
  arXiv  Detail & Related papers  (2026-07-13T02:11:27Z)
• REVES: REvision and VErification--Augmented Training for Test-Time Scaling [53.197756110943395]
  We propose a two-stage iterative framework that alternates between online data/prompt augmentation and policy optimization.<n>We observe gains of +6.5 points over the RL baseline and +4.0 points over standard multi-turn training.
  arXiv  Detail & Related papers  (2026-06-17T10:37:23Z)
• LimiX-2M: Mitigating Low-Rank Collapse and Attention Bottlenecks in Tabular Foundation Models [56.999481798138625]
  LimiX-2M is a 2M- parameter model that outperforms larger TabPFN-v2 and TabICL baselines on widely used benchmarks.<n>We present a unified tokenize-and-route framework for strong Tabular foundation models (TFMs)<n>Results highlight value-aware tokenization and readout-aligned routing as key levers for improving the accuracy--efficiency trade-off in TFMs.
  arXiv  Detail & Related papers  (2026-06-03T06:07:33Z)
• Learning from the Best: Smoothness-Driven Metrics for Data Quality in Imitation Learning [2.452887009229123]
  RINSE is a framework for scoring demonstrations based on trajectory smoothness that is policy-architecture-agnostic.<n>We show that smoothness filtering can reduce the conditional action variance of the retained data distribution.<n>As soft weights in Re-Mix domain reweighting, RINSE scores produce domain allocations highly correlated with the learned Re-Mix allocations.
  arXiv  Detail & Related papers  (2026-04-24T20:40:11Z)
• From $O(mn)$ to $O(r^2)$: Two-Sided Low-Rank Communication for Adam in Distributed Training with Memory Efficiency [28.885724420612323]
  We propose TSR, which brings two-sided low-rank communication to Adam-family updates (TSR-Adam)<n>To further reduce the peak communication from subspace refresh, TSR-Adam adopts a randomized SVD-based refresh that avoids full-gradient synchronization.<n>Across pretraining from 60M to 1B model scales, TSR-Adam reduces average communicated bytes per step by $13times$, and on GLUE fine-tuning it reduces communication by $25times$, while achieving comparable performance.
  arXiv  Detail & Related papers  (2026-02-08T15:23:09Z)
• One-Shot Federated Ridge Regression: Exact Recovery via Sufficient Statistic Aggregation [0.7106986689736825]
  Federated ridge regression is a distributed equilibrium problem where each client computes local sufficient statistics and transmits them once.<n>We establish differential privacy guarantees where noise is injected once per client, eliminating the composition penalty that degrades privacy in multi-round protocols.<n>Experiments on synthetic heterogeneous regression demonstrate that one-shot fusion matches FedAvg accuracy while requiring up to $38times$ less communication.
  arXiv  Detail & Related papers  (2026-01-13T04:47:22Z)
• FedScalar: A Communication efficient Federated Learning [0.0]
  Federated learning (FL) has gained considerable popularity for distributed machine learning. emphFedScalar enables agents to communicate updates using a single scalar.
  arXiv  Detail & Related papers  (2024-10-03T07:06:49Z)
• Generating Unbiased Pseudo-labels via a Theoretically Guaranteed Chebyshev Constraint to Unify Semi-supervised Classification and Regression [57.17120203327993]
  threshold-to-pseudo label process (T2L) in classification uses confidence to determine the quality of label. In nature, regression also requires unbiased methods to generate high-quality labels. We propose a theoretically guaranteed constraint for generating unbiased labels based on Chebyshev's inequality.
  arXiv  Detail & Related papers  (2023-11-03T08:39:35Z)
• Generalized Differentiable RANSAC [95.95627475224231]
  $nabla$-RANSAC is a differentiable RANSAC that allows learning the entire randomized robust estimation pipeline. $nabla$-RANSAC is superior to the state-of-the-art in terms of accuracy while running at a similar speed to its less accurate alternatives.
  arXiv  Detail & Related papers  (2022-12-26T15:13:13Z)
• OFedQIT: Communication-Efficient Online Federated Learning via Quantization and Intermittent Transmission [7.6058140480517356]
  Online federated learning (OFL) is a promising framework to collaboratively learn a sequence of non-linear functions (or models) from distributed streaming data. We propose a communication-efficient OFL algorithm (named OFedQIT) by means of a quantization and an intermittent transmission. Our analysis reveals that OFedQIT successfully addresses the drawbacks of OFedAvg while maintaining superior learning accuracy.
  arXiv  Detail & Related papers  (2022-05-13T07:46:43Z)
• Federated Optimization of Smooth Loss Functions [35.944772011421264]
  In this work, we study empirical risk minimization (ERM) within a federated learning framework. We propose the Federated Low Rank Gradient Descent (FedLRGD) algorithm. Our method solves the ERM problem at the server using inexact gradient descent.
  arXiv  Detail & Related papers  (2022-01-06T07:40:51Z)
• Improving Calibration for Long-Tailed Recognition [68.32848696795519]
  We propose two methods to improve calibration and performance in such scenarios. For dataset bias due to different samplers, we propose shifted batch normalization. Our proposed methods set new records on multiple popular long-tailed recognition benchmark datasets.
  arXiv  Detail & Related papers  (2021-04-01T13:55:21Z)
• SLOE: A Faster Method for Statistical Inference in High-Dimensional Logistic Regression [68.66245730450915]
  We develop an improved method for debiasing predictions and estimating frequentist uncertainty for practical datasets. Our main contribution is SLOE, an estimator of the signal strength with convergence guarantees that reduces the computation time of estimation and inference by orders of magnitude.
  arXiv  Detail & Related papers  (2021-03-23T17:48:56Z)
• Improving Semi-supervised Federated Learning by Reducing the Gradient Diversity of Models [67.66144604972052]
  Federated learning (FL) is a promising way to use the computing power of mobile devices while maintaining privacy of users. We show that a critical issue that affects the test accuracy is the large gradient diversity of the models from different users. We propose a novel grouping-based model averaging method to replace the FedAvg averaging method.
  arXiv  Detail & Related papers  (2020-08-26T03:36:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.