Winning the CityLearn Challenge: Adaptive Optimization with Evolutionary Search under Trajectory-based Guidance

• URL: http://arxiv.org/abs/2212.01939v1
• Date: Sun, 4 Dec 2022 22:18:38 GMT
• Title: Winning the CityLearn Challenge: Adaptive Optimization with Evolutionary Search under Trajectory-based Guidance
• Authors: Vanshaj Khattar and Ming Jin
• Abstract summary: We present a novel method using the solution function of optimization as policies to compute actions for sequential decision-making. Our agent ranked first in the latest 2021 CityLearn Challenge, being able to achieve superior performance in almost all metrics.
• Score: 2.4476800587391234
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern power systems will have to face difficult challenges in the years to come: frequent blackouts in urban areas caused by high power demand peaks, grid instability exacerbated by intermittent renewable generation, and global climate change amplified by rising carbon emissions. While current practices are growingly inadequate, the path to widespread adoption of artificial intelligence (AI) methods is hindered by missing aspects of trustworthiness. The CityLearn Challenge is an exemplary opportunity for researchers from multiple disciplines to investigate the potential of AI to tackle these pressing issues in the energy domain, collectively modeled as a reinforcement learning (RL) task. Multiple real-world challenges faced by contemporary RL techniques are embodied in the problem formulation. In this paper, we present a novel method using the solution function of optimization as policies to compute actions for sequential decision-making, while notably adapting the parameters of the optimization model from online observations. Algorithmically, this is achieved by an evolutionary algorithm under a novel trajectory-based guidance scheme. Formally, the global convergence property is established. Our agent ranked first in the latest 2021 CityLearn Challenge, being able to achieve superior performance in almost all metrics while maintaining some key aspects of interpretability.

Related papers

• Deep Reinforcement Learning for Online Optimal Execution Strategies [49.1574468325115]
  This paper tackles the challenge of learning non-Markovian optimal execution strategies in dynamic financial markets. We introduce a novel actor-critic algorithm based on Deep Deterministic Policy Gradient (DDPG) We show that our algorithm successfully approximates the optimal execution strategy.
  arXiv  Detail & Related papers  (2024-10-17T12:38:08Z)
• Aquatic Navigation: A Challenging Benchmark for Deep Reinforcement Learning [53.3760591018817]
  We propose a new benchmarking environment for aquatic navigation using recent advances in the integration between game engines and Deep Reinforcement Learning. Specifically, we focus on PPO, one of the most widely accepted algorithms, and we propose advanced training techniques. Our empirical evaluation shows that a well-designed combination of these ingredients can achieve promising results.
  arXiv  Detail & Related papers  (2024-05-30T23:20:23Z)
• Efficient Imitation Learning with Conservative World Models [54.52140201148341]
  We tackle the problem of policy learning from expert demonstrations without a reward function. We re-frame imitation learning as a fine-tuning problem, rather than a pure reinforcement learning one.
  arXiv  Detail & Related papers  (2024-05-21T20:53:18Z)
• Learning fast changing slow in spiking neural networks [3.069335774032178]
  Reinforcement learning (RL) faces substantial challenges when applied to real-life problems. Life-long learning machines must resolve the plasticity-stability paradox. Striking a balance between acquiring new knowledge and maintaining stability is crucial for artificial agents.
  arXiv  Detail & Related papers  (2024-01-25T12:03:10Z)
• Hybrid Reinforcement Learning for Optimizing Pump Sustainability in Real-World Water Distribution Networks [55.591662978280894]
  This article addresses the pump-scheduling optimization problem to enhance real-time control of real-world water distribution networks (WDNs) Our primary objectives are to adhere to physical operational constraints while reducing energy consumption and operational costs. Traditional optimization techniques, such as evolution-based and genetic algorithms, often fall short due to their lack of convergence guarantees.
  arXiv  Detail & Related papers  (2023-10-13T21:26:16Z)
• Self-Sustaining Multiple Access with Continual Deep Reinforcement Learning for Dynamic Metaverse Applications [17.436875530809946]
  The Metaverse is a new paradigm that aims to create a virtual environment consisting of numerous worlds, each of which will offer a different set of services. To deal with such a dynamic and complex scenario, one potential approach is to adopt self-sustaining strategies. This paper investigates the problem of multiple access in multi-channel environments to maximize the throughput of the intelligent agent.
  arXiv  Detail & Related papers  (2023-09-18T22:02:47Z)
• End-to-end Lidar-Driven Reinforcement Learning for Autonomous Racing [0.0]
  Reinforcement Learning (RL) has emerged as a transformative approach in the domains of automation and robotics. This study develops and trains an RL agent to navigate a racing environment solely using feedforward raw lidar and velocity data. The agent's performance is then experimentally evaluated in a real-world racing scenario.
  arXiv  Detail & Related papers  (2023-09-01T07:03:05Z)
• MARLIN: Soft Actor-Critic based Reinforcement Learning for Congestion Control in Real Networks [63.24965775030673]
  We propose a novel Reinforcement Learning (RL) approach to design generic Congestion Control (CC) algorithms. Our solution, MARLIN, uses the Soft Actor-Critic algorithm to maximize both entropy and return. We trained MARLIN on a real network with varying background traffic patterns to overcome the sim-to-real mismatch.
  arXiv  Detail & Related papers  (2023-02-02T18:27:20Z)
• Reinforcement Learning-Empowered Mobile Edge Computing for 6G Edge Intelligence [76.96698721128406]
  Mobile edge computing (MEC) considered a novel paradigm for computation and delay-sensitive tasks in fifth generation (5G) networks and beyond. This paper provides a comprehensive research review on free-enabled RL and offers insight for development.
  arXiv  Detail & Related papers  (2022-01-27T10:02:54Z)
• First Responders Got Wings: UAVs to the Rescue of Localization Operations in Beyond 5G Systems [7.244860161025552]
  Unmanned Aerial Vehicles (UAVs)-based solutions are the most promising candidates to take up on the localization challenge. In this paper, we capitalize on such recently available techniques by shedding light on the main challenges and future opportunities to boost the localization performance of state-of-the-art techniques.
  arXiv  Detail & Related papers  (2021-09-07T16:19:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.