Constraint-Aware Aggregation for Federated Reinforcement Learning in Microgrid Energy Coordination

• URL: http://arxiv.org/abs/2607.12763v1
• Date: Tue, 14 Jul 2026 13:33:25 GMT
• Title: Constraint-Aware Aggregation for Federated Reinforcement Learning in Microgrid Energy Coordination
• Abstract summary: Federated Reinforcement Learning (FedRL) enables coordination of distributed energy resources without sharing raw local data.<n>Standard aggregation methods such as FedAvg do not account for system-level constraints, often leading to unsafe global behavior.<n>We propose aggregation rules that incorporate both local performance and estimated constraint violation into the server-side update.
• Score: 1.675857332621569
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated Reinforcement Learning (FedRL) enables coordination of distributed energy resources without sharing raw local data, but standard aggregation methods such as FedAvg do not account for system-level constraints, often leading to unsafe global behavior. In this work, we study constraint-aware aggregation for federated reinforcement learning in distributed energy coordination. We propose aggregation rules that incorporate both local performance and estimated constraint violation into the server-side update. Among these, a simple penalty-based rule, $w_i \propto R_i - αV_i$, consistently provides the most reliable trade-off between reward and safety, without requiring dual optimization or modifications to local training. \textcolor{black}{We evaluate our approach on DairyGridEnv, a benchmark modeling multiple farms coordinating battery storage under stochastic demand and a shared grid capacity constraint, and further assess robustness using real load-driven demand profiles from Finland and the German FIELD dataset. Across multiple seeds, penalty-based aggregation substantially reduces violations while improving reward relative to FedAvg in both synthetic and real load-driven settings.} A combined reward-violation scheme exposes a tunable trade-off via $λ$, but is less stable. These results demonstrate that lightweight aggregation strategies can substantially improve empirical safety in federated reinforcement learning while preserving standard communication protocols.

Related papers

• Ratio-Variance Regularized Policy Optimization [64.95520246570446]
  We show that explicitly constraining the policy ratio variance provides a principled local approximation to trust-region constraints.<n>We introduce $bf R2bf VPO$ (Ratio-Variance Regularized Policy Optimization), which implements this constraint via a primal-dual optimization framework.
  arXiv  Detail & Related papers  (2026-05-26T09:53:42Z)
• An Efficient and Privacy-Preserving Architecture for Cross-Institutional Collaborative RAG [35.62196578084403]
  Retrieval-Augmented Generation empowers LLMs with external knowledge.<n>Strict privacy regulations create severe "data silos" that obstruct such collaboration.<n>We present FedRAG, a high- throughput, privacy-preserving federated RAG framework.
  arXiv  Detail & Related papers  (2026-05-25T11:18:19Z)
• Cooperative Deep Reinforcement Learning for Fair RIS Allocation [2.217288163160845]
  We consider RISs as shared infrastructure that must be dynamically assigned among competing base stations.<n>We propose a fairness-aware collaborative multi-agent reinforcement learning approach.<n>Results show that the proposed framework effectively redistributes RIS resources toward weaker-performing cells.
  arXiv  Detail & Related papers  (2026-03-26T15:46:32Z)
• Byzantine-Robust and Differentially Private Federated Optimization under Weaker Assumptions [89.52532304099522]
  Federated Learning (FL) enables clients to collaboratively train a shared model without centralizing their raw data, offering an inherent level of privacy.<n> gradients and model updates can still leak sensitive information, while malicious servers may mount adversarial attacks such as Byzantine manipulation.<n>These vulnerabilities highlight the need to address differential privacy (DP) and Byzantine robustness within a unified framework.<n>We propose Byz-Clip21-SGD2M, a new algorithm that integrates robust aggregation with double momentum and carefully designed clipping.
  arXiv  Detail & Related papers  (2026-03-24T17:39:09Z)
• Safe RLHF Beyond Expectation: Stochastic Dominance for Universal Spectral Risk Control [12.053369001333058]
  We propose a risk-sensitive alignment framework that replaces expected cost constraints with First-Order Dominance (FSD) constraints.<n>We show that RAD improves harmlessness over baselines while remaining competitive in helpfulness, and exhibits greater robustness on out-of-distribution evaluations.
  arXiv  Detail & Related papers  (2026-03-11T16:24:20Z)
• Compressed Proximal Federated Learning for Non-Convex Composite Optimization on Heterogeneous Data [12.4809003470908]
  FCO achieves communication efficiency and robustness when dealing with non-smoothizers.<n>Fed Composite Error FeedbackCEF is a novel algorithm for non-efficient quant convergence.<n>It mitigates the noise from aggressiveization and the noise from non-IID data.
  arXiv  Detail & Related papers  (2026-03-08T14:37:16Z)
• Unifying Stable Optimization and Reference Regularization in RLHF [64.16830602324345]
  This paper introduces a unified regularization approach that balances objectives of preventing reward hacking and maintaining stable policy updates.<n>Our simple yet principled alignment objective yields a weighted supervised fine-tuning loss with a superior trade-off, which demonstrably improves both alignment results and implementation complexity.
  arXiv  Detail & Related papers  (2026-02-12T03:31:19Z)
• Replacing Parameters with Preferences: Federated Alignment of Heterogeneous Vision-Language Models [63.70401095689976]
  We argue that replacing parameters with preferences represents a more scalable and privacy-preserving future.<n>We propose MoR, a federated alignment framework based on GRPO with Mixture-of-Rewards for heterogeneous VLMs.<n>MoR consistently outperforms federated alignment baselines in generalization, robustness, and cross-client adaptability.
  arXiv  Detail & Related papers  (2026-01-31T03:11:51Z)
• CoCo-Fed: A Unified Framework for Memory- and Communication-Efficient Federated Learning at the Wireless Edge [50.42067935605982]
  We propose a novel Compression and Combination-based Federated learning framework that unifies local memory efficiency and global communication reduction.<n>CoCo-Fed significantly outperforms state-of-the-art baselines in both memory and communication efficiency while maintaining robust convergence under non-IID settings.
  arXiv  Detail & Related papers  (2026-01-02T03:39:50Z)
• Entropy Ratio Clipping as a Soft Global Constraint for Stable Reinforcement Learning [49.92803982100042]
  We propose using the entropy ratio between the current and previous policies as a new global metric.<n>We introduce an textbfEntropy Ratio (ERC) mechanism that imposes bidirectional constraints on the entropy ratio.<n>This stabilizes policy updates at the global distribution level and compensates for the inability of PPO-clip to regulate probability shifts of un-sampled actions.
  arXiv  Detail & Related papers  (2025-12-05T10:26:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.