Fed-SE: Federated Self-Evolution for Privacy-Constrained Multi-Environment LLM Agents

• URL: http://arxiv.org/abs/2512.08870v1
• Date: Tue, 09 Dec 2025 18:04:41 GMT
• Title: Fed-SE: Federated Self-Evolution for Privacy-Constrained Multi-Environment LLM Agents
• Authors: Xiang Chen, Yuling Shi, Qizhen Lan, Yuchao Qiu, Xiaodong Gu,
• Abstract summary: We propose Fed-SE, a Federated Self-Evolution framework for LLM agents.<n>Fed-SE establishes a local evolution-global aggregation paradigm.<n>Globally, Fed-SE aggregates updates within a low-rank subspace that disentangles environment-specific dynamics.
• Score: 12.282703619791162
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: LLM agents are widely deployed in complex interactive tasks, yet privacy constraints often preclude centralized optimization and co-evolution across dynamic environments. While Federated Learning (FL) has proven effective on static datasets, its extension to the open-ended self-evolution of agents remains underexplored. Directly applying standard FL is challenging: heterogeneous tasks and sparse, trajectory-level rewards introduce severe gradient conflicts, destabilizing the global optimization process. To bridge this gap, we propose Fed-SE, a Federated Self-Evolution framework for LLM agents. Fed-SE establishes a local evolution-global aggregation paradigm. Locally, agents employ parameter-efficient fine-tuning on filtered, high-return trajectories to achieve stable gradient updates. Globally, Fed-SE aggregates updates within a low-rank subspace that disentangles environment-specific dynamics, effectively reducing negative transfer across clients. Experiments across five heterogeneous environments demonstrate that Fed-SE improves average task success rates by approximately 18% over federated baselines, validating its effectiveness in robust cross-environment knowledge transfer in privacy-constrained deployments.

Related papers

• FedZMG: Efficient Client-Side Optimization in Federated Learning [0.19116784879310023]
  Federated Zero Mean Gradients (FedZMG) is a parameter-free, client-side optimization algorithm designed to tackle client-drift.<n>FedZMG projects local gradients onto a zero-mean hyperplane, effectively neutralizing the "intensity" or "bias" shifts inherent in heterogeneous data distributions.
  arXiv  Detail & Related papers  (2026-02-20T17:45:28Z)
• Roughness-Informed Federated Learning [3.8218584696400484]
  Federated Learning (FL) enables collaborative model training across distributed clients.<n>FL faces challenges in non-independent and identically distributed (non-IID) settings due to client drift.<n>We propose RI-FedAvg, a novel FL that mitigates client drift by incorporating a Roughness Index (RI)-based regularization term.
  arXiv  Detail & Related papers  (2026-02-11T07:35:45Z)
• Adaptive Dual-Weighting Framework for Federated Learning via Out-of-Distribution Detection [53.45696787935487]
  Federated Learning (FL) enables collaborative model training across large-scale distributed service nodes.<n>In real-world service-oriented deployments, data generated by heterogeneous users, devices, and application scenarios are inherently non-IID.<n>We propose FLood, a novel FL framework inspired by out-of-distribution (OOD) detection.
  arXiv  Detail & Related papers  (2026-02-01T05:54:59Z)
• MAESTRO: Meta-learning Adaptive Estimation of Scalarization Trade-offs for Reward Optimization [56.074760766965085]
  Group-Relative Policy Optimization has emerged as an efficient paradigm for aligning Large Language Models (LLMs)<n>We propose MAESTRO, which treats reward scalarization as a dynamic latent policy, leveraging the model's terminal hidden states as a semantic bottleneck.<n>We formulate this as a contextual bandit problem within a bi-level optimization framework, where a lightweight Conductor network co-evolves with the policy by utilizing group-relative advantages as a meta-reward signal.
  arXiv  Detail & Related papers  (2026-01-12T05:02:48Z)
• Local Gradient Regulation Stabilizes Federated Learning under Client Heterogeneity [41.890949680958805]
  Federated learning (FL) enables collaborative model training across distributed clients without sharing raw data.<n>Clients destabilize FL by distorting local gradient dynamics during client-side optimization.<n>We develop a general client-side perspective that regulates local gradient contributions without incurring additional communication overhead.
  arXiv  Detail & Related papers  (2026-01-07T04:58:18Z)
• Alignment Tipping Process: How Self-Evolution Pushes LLM Agents Off the Rails [103.05296856071931]
  We identify the Alignment Tipping Process (ATP), a critical post-deployment risk unique to self-evolving Large Language Model (LLM) agents.<n>ATP arises when continual interaction drives agents to abandon alignment constraints established during training in favor of reinforced, self-interested strategies.<n>Our experiments show that alignment benefits erode rapidly under self-evolution, with initially aligned models converging toward unaligned states.
  arXiv  Detail & Related papers  (2025-10-06T14:48:39Z)
• Federated Reinforcement Learning in Heterogeneous Environments [9.944647907864255]
  We investigate a Federated Reinforcement Learning with Environment Heterogeneity (FRL-EH) framework, where local environments exhibit statistical heterogeneity.<n>Within this framework, agents collaboratively learn a global policy by aggregating their collective experiences while preserving the privacy of their local trajectories.<n>We present a novel global objective function that ensures robust performance across heterogeneous local environments and their plausible perturbations.<n>We extend FedRQ to environments with continuous state space through the use of expectile loss, addressing the key challenge of minimizing a value function over a continuous subset of the state space.
  arXiv  Detail & Related papers  (2025-07-19T05:06:38Z)
• Generalized and Personalized Federated Learning with Foundation Models via Orthogonal Transformations [4.008780119020479]
  Federated Learning aims to train models across decentralized clients or devices holding local data without the need for centralized data collection.<n>We introduce FedOT, a novel approach that leverages black-box foundation models.<n>FedOT mitigates gradient conflicts across diverse clients, preserves semantic integrity, and achieves robust performance even in the presence of substantial data.
  arXiv  Detail & Related papers  (2025-05-26T12:18:24Z)
• Towards Robust and Efficient Federated Low-Rank Adaptation with Heterogeneous Clients [6.570712059945705]
  Low Rank Adaptation (LoRA) has been proposed as a solution, yet its application in federated learning is complicated by discordance in aggregation.<n>Existing methods addressing this discordance often suffer from performance degradation at low ranks in heterogeneous data settings.<n>We introduce LoRA-A$2$, which demonstrates robustness in challenging settings with low ranks and high data heterogeneity.
  arXiv  Detail & Related papers  (2024-10-30T08:48:21Z)
• FedEGG: Federated Learning with Explicit Global Guidance [90.04705121816185]
  Federated Learning (FL) holds great potential for diverse applications owing to its privacy-preserving nature.<n>Existing methods help address these challenges via optimization-based client constraints, adaptive client selection, or the use of pre-trained models or synthetic data.<n>We present bftextFedEGG, a new FL algorithm that constructs a global guiding task using a well-defined, easy-to-converge learning task.
  arXiv  Detail & Related papers  (2024-04-18T04:25:21Z)
• Federated Natural Policy Gradient and Actor Critic Methods for Multi-task Reinforcement Learning [46.28771270378047]
  Federated reinforcement learning (RL) enables collaborative decision making of multiple distributed agents without sharing local data trajectories. In this work, we consider a multi-task setting, in which each agent has its own private reward function corresponding to different tasks, while sharing the same transition kernel of the environment. We learn a globally optimal policy that maximizes the sum of the discounted total rewards of all the agents in a decentralized manner.
  arXiv  Detail & Related papers  (2023-11-01T00:15:18Z)
• Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscape [59.841889495864386]
  In federated learning (FL), a cluster of local clients are chaired under the coordination of a global server. Clients are prone to overfit into their own optima, which extremely deviates from the global objective. ttfamily FedSMOO adopts a dynamic regularizer to guarantee the local optima towards the global objective. Our theoretical analysis indicates that ttfamily FedSMOO achieves fast $mathcalO (1/T)$ convergence rate with low bound generalization.
  arXiv  Detail & Related papers  (2023-05-19T10:47:44Z)
• Disentangled Federated Learning for Tackling Attributes Skew via Invariant Aggregation and Diversity Transferring [104.19414150171472]
  Attributes skews the current federated learning (FL) frameworks from consistent optimization directions among the clients. We propose disentangled federated learning (DFL) to disentangle the domain-specific and cross-invariant attributes into two complementary branches. Experiments verify that DFL facilitates FL with higher performance, better interpretability, and faster convergence rate, compared with SOTA FL methods.
  arXiv  Detail & Related papers  (2022-06-14T13:12:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.