Online Learning and Distributed Control for Residential Demand Response

• URL: http://arxiv.org/abs/2010.05153v2
• Date: Sat, 12 Jun 2021 02:19:54 GMT
• Title: Online Learning and Distributed Control for Residential Demand Response
• Authors: Xin Chen, Yingying Li, Jun Shimada, Na Li
• Abstract summary: This paper studies the automated control method for regulating air conditioner (AC) loads in incentive-based residential demand response (DR) We formulate the AC control problem in a DR event as a multi-period transition optimization that integrates the indoor thermal dynamics and customer opt-out status. We propose an online DR control algorithm to learn customer behaviors and make real-time AC control schemes.
• Score: 16.61679791774638
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: This paper studies the automated control method for regulating air conditioner (AC) loads in incentive-based residential demand response (DR). The critical challenge is that the customer responses to load adjustment are uncertain and unknown in practice. In this paper, we formulate the AC control problem in a DR event as a multi-period stochastic optimization that integrates the indoor thermal dynamics and customer opt-out status transition. Specifically, machine learning techniques including Gaussian process and logistic regression are employed to learn the unknown thermal dynamics model and customer opt-out behavior model, respectively. We consider two typical DR objectives for AC load control: 1) minimizing the total demand, 2) closely tracking a regulated power trajectory. Based on the Thompson sampling framework, we propose an online DR control algorithm to learn customer behaviors and make real-time AC control schemes. This algorithm considers the influence of various environmental factors on customer behaviors and is implemented in a distributed fashion to preserve the privacy of customers. Numerical simulations demonstrate the control optimality and learning efficiency of the proposed algorithm.

Related papers

• Scenario-based Thermal Management Parametrization Through Deep Reinforcement Learning [0.4218593777811082]
  This paper introduces a learning-based tuning approach for thermal management functions. Our deep reinforcement learning agent processes the tuning task context and incorporates an image-based interpretation of embedded parameter sets. We demonstrate its applicability to a valve controller parametrization task and verify it in real-world vehicle testing.
  arXiv  Detail & Related papers  (2024-08-04T13:19:45Z)
• Growing Q-Networks: Solving Continuous Control Tasks with Adaptive Control Resolution [51.83951489847344]
  In robotics applications, smooth control signals are commonly preferred to reduce system wear and energy efficiency. In this work, we aim to bridge this performance gap by growing discrete action spaces from coarse to fine control resolution. Our work indicates that an adaptive control resolution in combination with value decomposition yields simple critic-only algorithms that yield surprisingly strong performance on continuous control tasks.
  arXiv  Detail & Related papers  (2024-04-05T17:58:37Z)
• Deployable Reinforcement Learning with Variable Control Rate [14.838483990647697]
  We propose a variant of Reinforcement Learning (RL) with variable control rate. In this approach, the policy decides the action the agent should take as well as the duration of the time step associated with that action. We show the efficacy of SEAC through a proof-of-concept simulation driving an agent with Newtonian kinematics.
  arXiv  Detail & Related papers  (2024-01-17T15:40:11Z)
• Incorporating Recurrent Reinforcement Learning into Model Predictive Control for Adaptive Control in Autonomous Driving [11.67417895998434]
  Model Predictive Control (MPC) is attracting tremendous attention in the autonomous driving task as a powerful control technique. In this paper, we reformulate the problem as a Partially Observed Markov Decision Process (POMDP) We then learn a recurrent policy continually adapting the parameters of the dynamics model via Recurrent Reinforcement Learning (RRL) for optimal and adaptive control.
  arXiv  Detail & Related papers  (2023-01-30T22:11:07Z)
• Policy Search for Model Predictive Control with Application to Agile Drone Flight [56.24908013905407]
  We propose a policy-search-for-model-predictive-control framework for MPC. Specifically, we formulate the MPC as a parameterized controller, where the hard-to-optimize decision variables are represented as high-level policies. Experiments show that our controller achieves robust and real-time control performance in both simulation and the real world.
  arXiv  Detail & Related papers  (2021-12-07T17:39:24Z)
• Neural optimal feedback control with local learning rules [67.5926699124528]
  A major problem in motor control is understanding how the brain plans and executes proper movements in the face of delayed and noisy stimuli. We introduce a novel online algorithm which combines adaptive Kalman filtering with a model free control approach.
  arXiv  Detail & Related papers  (2021-11-12T20:02:00Z)
• Reinforcement Learning for Thermostatically Controlled Loads Control using Modelica and Python [0.0]
  The project aims to investigate and assess opportunities for applying reinforcement learning (RL) for power system control. As a proof of concept (PoC), voltage control of thermostatically controlled loads (TCLs) for power consumption was developed using Modelica-based pipeline. The paper shows the influence of Q-learning parameters, including discretization of state-action space, on the controller performance.
  arXiv  Detail & Related papers  (2020-05-09T13:35:49Z)
• Logarithmic Regret Bound in Partially Observable Linear Dynamical Systems [91.43582419264763]
  We study the problem of system identification and adaptive control in partially observable linear dynamical systems. We present the first model estimation method with finite-time guarantees in both open and closed-loop system identification. We show that AdaptOn is the first algorithm that achieves $textpolylogleft(Tright)$ regret in adaptive control of unknown partially observable linear dynamical systems.
  arXiv  Detail & Related papers  (2020-03-25T06:00:33Z)
• Online Residential Demand Response via Contextual Multi-Armed Bandits [8.817815952311676]
  One major challenge in residential demand response (DR) is to handle the unknown and uncertain customer behaviors. Previous works use learning techniques to predict customer DR behaviors, while the influence of time-varying environmental factors is generally neglected. In this paper, we consider the residential DR problem where the load service entity (LSE) aims to select an optimal subset of customers to maximize the expected load reduction with a financial budget. Online learning and selection (OLS) algorithm based on Thompson sampling is proposed to solve it.
  arXiv  Detail & Related papers  (2020-03-07T18:17:50Z)
• NeurOpt: Neural network based optimization for building energy management and climate control [58.06411999767069]
  We propose a data-driven control algorithm based on neural networks to reduce this cost of model identification. We validate our learning and control algorithms on a two-story building with ten independently controlled zones, located in Italy.
  arXiv  Detail & Related papers  (2020-01-22T00:51:03Z)
• Information Theoretic Model Predictive Q-Learning [64.74041985237105]
  We present a novel theoretical connection between information theoretic MPC and entropy regularized RL. We develop a Q-learning algorithm that can leverage biased models.
  arXiv  Detail & Related papers  (2019-12-31T00:29:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.