Federated Nested Learning: Collaborative Training of Self-Referential Memories for Test-Time Adaptation

• URL: http://arxiv.org/abs/2605.16350v1
• Date: Fri, 08 May 2026 04:31:46 GMT
• Title: Federated Nested Learning: Collaborative Training of Self-Referential Memories for Test-Time Adaptation
• Abstract summary: Federated Nested Learning (FedNL) is a novel framework that reformulates Federated Learning as a three-level nested optimization system.<n>FedNL embeds Titans-based linear attention into FL, enabling clients to perform lightweight, zero-shot test-time adaptation.
• Score: 50.940432795341735
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We rethink Federated Learning (FL) from a nested learning perspective, framing the core challenge as how to collaboratively learn optimization rules, not just static models, to tackle Non-IID client data. To address this, we propose Federated Nested Learning (FedNL), a novel framework that reformulates FL as a three-level nested optimization system. FedNL embeds Titans-based linear attention into FL, enabling clients to perform lightweight, zero-shot test-time adaptation by treating a delta rule as an online gradient step. Experiments on Non-IID MMLU and long-context benchmarks show that FedNL achieves competitive performance in short-context reasoning, enhances the performance of long-context retrieval and streaming Cross-Entropy, and maintains constant inference memory.

Related papers

• Dynamic Participation in Federated Learning: Benchmarks and a Knowledge Pool Plugin [10.912739346462525]
  Federated learning (FL) enables clients to collaboratively train a shared model in a distributed manner.<n>Most existing FL research assumes consistent client participation, overlooking the practical scenario of dynamic participation.<n>We present the first open-source framework explicitly designed for benchmarking FL models under dynamic client participation.
  arXiv  Detail & Related papers  (2025-11-20T16:36:50Z)
• On the Convergence of Continual Federated Learning Using Incrementally Aggregated Gradients [7.226144684379189]
  The holy grail of machine learning is to enable Continual Federated Learning (CFL) to enhance the efficiency, privacy, and scalability of AI systems while learning from streaming data.<n>We propose a novel replay-memory based federated strategy consisting of edge-based gradient updates on memory and aggregated gradients on the current data.<n>We empirically show that C-FLAG outperforms several state-of-the-art baselines on both task and class-incremental settings with respect to metrics such as accuracy and forgetting.
  arXiv  Detail & Related papers  (2024-11-12T17:36:20Z)
• FADAS: Towards Federated Adaptive Asynchronous Optimization [56.09666452175333]
  Federated learning (FL) has emerged as a widely adopted training paradigm for privacy-preserving machine learning. This paper introduces federated adaptive asynchronous optimization, named FADAS, a novel method that incorporates asynchronous updates into adaptive federated optimization with provable guarantees. We rigorously establish the convergence rate of the proposed algorithms and empirical results demonstrate the superior performance of FADAS over other asynchronous FL baselines.
  arXiv  Detail & Related papers  (2024-07-25T20:02:57Z)
• FedEGG: Federated Learning with Explicit Global Guidance [90.04705121816185]
  Federated Learning (FL) holds great potential for diverse applications owing to its privacy-preserving nature.<n>Existing methods help address these challenges via optimization-based client constraints, adaptive client selection, or the use of pre-trained models or synthetic data.<n>We present bftextFedEGG, a new FL algorithm that constructs a global guiding task using a well-defined, easy-to-converge learning task.
  arXiv  Detail & Related papers  (2024-04-18T04:25:21Z)
• Relaxed Contrastive Learning for Federated Learning [48.96253206661268]
  We propose a novel contrastive learning framework to address the challenges of data heterogeneity in federated learning. Our framework outperforms all existing federated learning approaches by huge margins on the standard benchmarks.
  arXiv  Detail & Related papers  (2024-01-10T04:55:24Z)
• Knowledge Rumination for Client Utility Evaluation in Heterogeneous Federated Learning [12.50871784200551]
  Federated Learning (FL) allows several clients to cooperatively train machine learning models without disclosing the raw data.<n>Non-IID data and stale models pose significant challenges to AFL, as they can diminish the practicality of the global model and even lead to training failures.<n>We propose a novel AFL framework called Federated Historical Learning (FedHist), which effectively addresses the challenges posed by both Non-IID data and gradient staleness.
  arXiv  Detail & Related papers  (2023-12-16T11:40:49Z)
• FedLALR: Client-Specific Adaptive Learning Rates Achieve Linear Speedup for Non-IID Data [54.81695390763957]
  Federated learning is an emerging distributed machine learning method. We propose a heterogeneous local variant of AMSGrad, named FedLALR, in which each client adjusts its learning rate. We show that our client-specified auto-tuned learning rate scheduling can converge and achieve linear speedup with respect to the number of clients.
  arXiv  Detail & Related papers  (2023-09-18T12:35:05Z)
• Scheduling and Aggregation Design for Asynchronous Federated Learning over Wireless Networks [56.91063444859008]
  Federated Learning (FL) is a collaborative machine learning framework that combines on-device training and server-based aggregation. We propose an asynchronous FL design with periodic aggregation to tackle the straggler issue in FL systems. We show that an age-aware'' aggregation weighting design can significantly improve the learning performance in an asynchronous FL setting.
  arXiv  Detail & Related papers  (2022-12-14T17:33:01Z)
• Towards Federated Learning on Time-Evolving Heterogeneous Data [13.080665001587281]
  Federated Learning (FL) is an emerging learning paradigm that preserves privacy by ensuring client data locality on edge devices. Despite recent research efforts on improving the optimization of heterogeneous data, the impact of time-evolving heterogeneous data in real-world scenarios has not been well studied. We propose Continual Federated Learning (CFL), a flexible framework, to capture the time-evolving heterogeneity of FL.
  arXiv  Detail & Related papers  (2021-12-25T14:58:52Z)
• Multiple Kernel-Based Online Federated Learning [15.102346715690755]
  We propose a novel multiple kernel-based OFL (MK-OFL) as a non-trivial extension of OMKL. We theoretically prove that MK-OFL achieves the optimal sublinear regret bound when compared with the best function in hindsight.
  arXiv  Detail & Related papers  (2021-02-22T09:51:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.