Deep Reinforcement Learning-Assisted Federated Learning for Robust Short-term Utility Demand Forecasting in Electricity Wholesale Markets

• URL: http://arxiv.org/abs/2206.11715v2
• Date: Tue, 25 Jul 2023 16:18:12 GMT
• Title: Deep Reinforcement Learning-Assisted Federated Learning for Robust Short-term Utility Demand Forecasting in Electricity Wholesale Markets
• Abstract summary: Short-term load forecasting (STLF) plays a significant role in the operation of electricity trading markets. In wholesale markets, as it is not realistic for power plants (PPs) to access UCs' data directly, federated learning (FL) is definitely a feasible solution. We propose a DRL-assisted FL approach, DEfect-AwaRe federated soft actor-critic (DearFSAC) to robustly train an accurate STLF model for PPs.
• Score: 3.631233287824939
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Short-term load forecasting (STLF) plays a significant role in the operation of electricity trading markets. Considering the growing concern of data privacy, federated learning (FL) is increasingly adopted to train STLF models for utility companies (UCs) in recent research. Inspiringly, in wholesale markets, as it is not realistic for power plants (PPs) to access UCs' data directly, FL is definitely a feasible solution of obtaining an accurate STLF model for PPs. However, due to FL's distributed nature and intense competition among UCs, defects increasingly occur and lead to poor performance of the STLF model, indicating that simply adopting FL is not enough. In this paper, we propose a DRL-assisted FL approach, DEfect-AwaRe federated soft actor-critic (DearFSAC), to robustly train an accurate STLF model for PPs to forecast precise short-term utility electricity demand. Firstly. we design a STLF model based on long short-term memory (LSTM) using just historical load data and time data. Furthermore, considering the uncertainty of defects occurrence, a deep reinforcement learning (DRL) algorithm is adopted to assist FL by alleviating model degradation caused by defects. In addition, for faster convergence of FL training, an auto-encoder is designed for both dimension reduction and quality evaluation of uploaded models. In the simulations, we validate our approach on real data of Helsinki's UCs in 2019. The results show that DearFSAC outperforms all the other approaches no matter if defects occur or not.

Related papers

• Entropy-KL Divergence-based Token Masking: A Novel Approach for Selective Fine-tuning of Large Language Models [52.11240605311707]
  Supervised fine-tuning followed by reinforcement learning has become a standard post-training paradigm for large language models.<n>We propose EKSFT (Entropy-KL Selective Fine-Tuning), which selectively masks tokens that exhibit either high entropy or high KL divergence from a reference model.<n> Empirical evaluations on mathematical reasoning benchmarks demonstrate that EKSFT consistently outperforms standard SFT.
  arXiv  Detail & Related papers  (2026-05-28T03:36:05Z)
• Trust-Region Adaptive Policy Optimization [82.09255251747818]
  Post-training methods play an important role in improving large language models' (LLMs) complex reasoning abilities.<n>We introduce TRAPO, a framework that interleavesSupervised Fine-Tuning (SFT) and Reinforcement Learning (RL) within each training instance.<n>Experiments on five mathematical reasoning benchmarks show that TRAPO consistently surpasses standard SFT, RL, and SFT-then-RL pipelines.
  arXiv  Detail & Related papers  (2025-12-19T14:37:07Z)
• Feasible Learning [78.6167929413604]
  We introduce Feasible Learning (FL), a sample-centric learning paradigm where models are trained by solving a feasibility problem that bounds the loss for each training sample. Our empirical analysis, spanning image classification, age regression, and preference optimization in large language models, demonstrates that models trained via FL can learn from data while displaying improved tail behavior compared to ERM, with only a marginal impact on average performance.
  arXiv  Detail & Related papers  (2025-01-24T20:39:38Z)
• Unlocking the Potential of Model Calibration in Federated Learning [15.93119575317457]
  We propose Non-Uniform for Federated Learning (NUCFL), a generic framework that integrates FL with the concept of model calibration. OurFL addresses this challenge by dynamically adjusting the model calibration based on relationships between each clients local model and the global model in FL. By doing so,FL effectively aligns calibration needs for the global model while not sacrificing accuracy.
  arXiv  Detail & Related papers  (2024-09-07T20:11:11Z)
• SPD-CFL: Stepwise Parameter Dropout for Efficient Continual Federated Learning [18.917283498639442]
  We propose the Step-wise.<n>Dropout for Continual Federated Learning (SPD-CFL) approach.<n>It allows users to specify the target level of performance and then attempts to find the most suitable dropout rate for the given FL model.<n>It achieves a 2.07% higher test AUC while reducing communication overhead by 29.53%.
  arXiv  Detail & Related papers  (2024-05-15T14:50:46Z)
• Addressing Heterogeneity in Federated Load Forecasting with Personalization Layers [3.933147844455233]
  We propose the use of personalization layers for load forecasting in a general framework called PL-FL. We show that PL-FL outperforms FL and purely local training, while requiring lower communication bandwidth than FL.
  arXiv  Detail & Related papers  (2024-04-01T22:53:09Z)
• Semi-Federated Learning: Convergence Analysis and Optimization of A Hybrid Learning Framework [70.83511997272457]
  We propose a semi-federated learning (SemiFL) paradigm to leverage both the base station (BS) and devices for a hybrid implementation of centralized learning (CL) and FL. We propose a two-stage algorithm to solve this intractable problem, in which we provide the closed-form solutions to the beamformers.
  arXiv  Detail & Related papers  (2023-10-04T03:32:39Z)
• Adaptive Model Pruning and Personalization for Federated Learning over Wireless Networks [72.59891661768177]
  Federated learning (FL) enables distributed learning across edge devices while protecting data privacy. We consider a FL framework with partial model pruning and personalization to overcome these challenges. This framework splits the learning model into a global part with model pruning shared with all devices to learn data representations and a personalized part to be fine-tuned for a specific device.
  arXiv  Detail & Related papers  (2023-09-04T21:10:45Z)
• Communication Resources Constrained Hierarchical Federated Learning for End-to-End Autonomous Driving [67.78611905156808]
  This paper proposes an optimization-based Communication Resource Constrained Hierarchical Federated Learning framework. Results show that the proposed CRCHFL both accelerates the convergence rate and enhances the generalization of federated learning autonomous driving model.
  arXiv  Detail & Related papers  (2023-06-28T12:44:59Z)
• Automated Federated Learning in Mobile Edge Networks -- Fast Adaptation and Convergence [83.58839320635956]
  Federated Learning (FL) can be used in mobile edge networks to train machine learning models in a distributed manner. Recent FL has been interpreted within a Model-Agnostic Meta-Learning (MAML) framework, which brings FL significant advantages in fast adaptation and convergence over heterogeneous datasets. This paper addresses how much benefit MAML brings to FL and how to maximize such benefit over mobile edge networks.
  arXiv  Detail & Related papers  (2023-03-23T02:42:10Z)
• A comparative assessment of deep learning models for day-ahead load forecasting: Investigating key accuracy drivers [2.572906392867547]
  Short-term load forecasting (STLF) is vital for the effective and economic operation of power grids and energy markets. Several deep learning models have been proposed in the literature for STLF, reporting promising results.
  arXiv  Detail & Related papers  (2023-02-23T17:11:04Z)
• Performance Optimization for Variable Bitwidth Federated Learning in Wireless Networks [103.22651843174471]
  This paper considers improving wireless communication and computation efficiency in federated learning (FL) via model quantization. In the proposed bitwidth FL scheme, edge devices train and transmit quantized versions of their local FL model parameters to a coordinating server, which aggregates them into a quantized global model and synchronizes the devices. We show that the FL training process can be described as a Markov decision process and propose a model-based reinforcement learning (RL) method to optimize action selection over iterations.
  arXiv  Detail & Related papers  (2022-09-21T08:52:51Z)
• DearFSAC: An Approach to Optimizing Unreliable Federated Learning via Deep Reinforcement Learning [3.516494777812123]
  We propose the DEfect-AwaRe federated soft actor-critic (DearFSAC) to dynamically assign weights to local models to improve the robustness of federated learning. DearFSAC outperforms three existing approaches on four datasets for both independent and identically distributed (IID) and non-IID settings.
  arXiv  Detail & Related papers  (2022-01-30T01:47:43Z)
• Federated Unlearning [24.60965999954735]
  Federated learning (FL) has emerged as a promising distributed machine learning (ML) paradigm. Practical needs of the "right to be forgotten" and countering data poisoning attacks call for efficient techniques that can remove, or unlearn, specific training data from the trained FL model. We present FedEraser, the first federated unlearning methodology that can eliminate the influence of a federated client's data on the global FL model.
  arXiv  Detail & Related papers  (2020-12-27T08:54:37Z)
• Delay Minimization for Federated Learning Over Wireless Communication Networks [172.42768672943365]
  The problem of delay computation for federated learning (FL) over wireless communication networks is investigated. A bisection search algorithm is proposed to obtain the optimal solution. Simulation results show that the proposed algorithm can reduce delay by up to 27.3% compared to conventional FL methods.
  arXiv  Detail & Related papers  (2020-07-05T19:00:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.