Prospect Theory-inspired Automated P2P Energy Trading with Q-learning-based Dynamic Pricing

• URL: http://arxiv.org/abs/2208.12777v1
• Date: Fri, 26 Aug 2022 16:45:40 GMT
• Title: Prospect Theory-inspired Automated P2P Energy Trading with Q-learning-based Dynamic Pricing
• Authors: Ashutosh Timilsina, Simone Silvestri
• Abstract summary: In this paper, we design an automated P2P energy market that takes user perception into account. We introduce a risk-sensitive Q-learning mechanism named Q-b Pricing and Risk-sensitivity (PQR), which learns the optimal price for sellers considering their perceived utility. Results based on real traces of energy consumption and production, as well as realistic prospect theory functions, show that our approach achieves a 26% higher perceived value for buyers.
• Score: 2.2463154358632473
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The widespread adoption of distributed energy resources, and the advent of smart grid technologies, have allowed traditionally passive power system users to become actively involved in energy trading. Recognizing the fact that the traditional centralized grid-driven energy markets offer minimal profitability to these users, recent research has shifted focus towards decentralized peer-to-peer (P2P) energy markets. In these markets, users trade energy with each other, with higher benefits than buying or selling to the grid. However, most researches in P2P energy trading largely overlook the user perception in the trading process, assuming constant availability, participation, and full compliance. As a result, these approaches may result in negative attitudes and reduced engagement over time. In this paper, we design an automated P2P energy market that takes user perception into account. We employ prospect theory to model the user perception and formulate an optimization framework to maximize the buyer's perception while matching demand and production. Given the non-linear and non-convex nature of the optimization problem, we propose Differential Evolution-based Algorithm for Trading Energy called DEbATE. Additionally, we introduce a risk-sensitive Q-learning algorithm, named Pricing mechanism with Q-learning and Risk-sensitivity (PQR), which learns the optimal price for sellers considering their perceived utility. Results based on real traces of energy consumption and production, as well as realistic prospect theory functions, show that our approach achieves a 26% higher perceived value for buyers and generates 7% more reward for sellers, compared to a recent state of the art approach.

Related papers

• Evaluating the Impact of Multiple DER Aggregators on Wholesale Energy Markets: A Hybrid Mean Field Approach [2.0535683313855055]
  The integration of distributed energy resources into wholesale energy markets can greatly enhance grid flexibility, improve market efficiency, and contribute to a more sustainable energy future. We study a wholesale market model featuring multiple DER aggregators, each controlling a portfolio of DER resources and bidding into the market on behalf of the DER asset owners. We propose a reinforcement learning (RL)-based method to help each agent learn optimal strategies within the MFG framework, enhancing their ability to adapt to market conditions and uncertainties.
  arXiv  Detail & Related papers  (2024-08-27T14:56:28Z)
• Peer-to-Peer Energy Trading of Solar and Energy Storage: A Networked Multiagent Reinforcement Learning Approach [5.671124014371425]
  We propose multi-agent reinforcement learning (MARL) frameworks to help automate consumers' bidding and management of their solar PV and energy storage resources. We show how the MARL frameworks can integrate physical network constraints to realize voltage control, hence ensuring physical feasibility of the P2P energy trading.
  arXiv  Detail & Related papers  (2024-01-25T05:05:55Z)
• Decentralized Energy Marketplace via NFTs and AI-based Agents [4.149465156450793]
  The paper introduces an advanced Decentralized Energy Marketplace (DEM) integrating blockchain technology and artificial intelligence. The proposed framework uses Non-Fungible Tokens (NFTs) to represent unique energy profiles in a transparent and secure trading environment. A notable innovation is the use of smart contracts, ensuring high efficiency and integrity in energy transactions.
  arXiv  Detail & Related papers  (2023-11-17T09:15:43Z)
• Energy Pricing in P2P Energy Systems Using Reinforcement Learning [36.244907785240876]
  The increase in renewable energy on the consumer side gives place to new dynamics in the energy grids. In such a scenario, the nature of distributed renewable energy generators and energy consumption increases the complexity of defining fair prices for buying and selling energy. We introduce a reinforcement learning framework to help solve this issue by training an agent to set the prices that maximize the profit of all components in the microgrid.
  arXiv  Detail & Related papers  (2022-10-24T19:21:10Z)
• Applications of blockchain and artificial intelligence technologies for enabling prosumers in smart grids: A review [12.609078866334615]
  Governments' net zero emission target aims at increasing the share of renewable energy sources. This paper addresses how to incorporate the blockchain and AI in the smart grids for facilitating prosumers to participate in energy markets.
  arXiv  Detail & Related papers  (2022-02-21T10:27:31Z)
• A Multi-Agent Deep Reinforcement Learning Approach for a Distributed Energy Marketplace in Smart Grids [58.666456917115056]
  This paper presents a Reinforcement Learning based energy market for a prosumer dominated microgrid. The proposed market model facilitates a real-time and demanddependent dynamic pricing environment, which reduces grid costs and improves the economic benefits for prosumers.
  arXiv  Detail & Related papers  (2020-09-23T02:17:51Z)
• Challenges and Prospects of Negawatt Trading in Light of Recent Technological Developments [69.98112767404695]
  We review the challenges and prospects of negawatt trading in light of recent technological advancements. Grid interactive buildings and distributed ledger technologies can ensure active participation and fair pricing.
  arXiv  Detail & Related papers  (2020-07-16T13:41:14Z)
• A Deep Reinforcement Learning Framework for Continuous Intraday Market Bidding [69.37299910149981]
  A key component for the successful renewable energy sources integration is the usage of energy storage. We propose a novel modelling framework for the strategic participation of energy storage in the European continuous intraday market. An distributed version of the fitted Q algorithm is chosen for solving this problem due to its sample efficiency. Results indicate that the agent converges to a policy that achieves in average higher total revenues than the benchmark strategy.
  arXiv  Detail & Related papers  (2020-04-13T13:50:13Z)
• Towards a Peer-to-Peer Energy Market: an Overview [68.8204255655161]
  This work focuses on the electric power market, comparing the status quo with the recent trend towards the increase in distributed self-generation capabilities by prosumers. We introduce a potential multi-layered architecture for a Peer-to-Peer (P2P) energy market, discussing the fundamental aspects of local production and local consumption as part of a microgrid. To give a full picture to the reader, we also scrutinise relevant elements of energy trading, such as Smart Contract and grid stability.
  arXiv  Detail & Related papers  (2020-03-02T20:32:10Z)
• Risk-Aware Energy Scheduling for Edge Computing with Microgrid: A Multi-Agent Deep Reinforcement Learning Approach [82.6692222294594]
  We study a risk-aware energy scheduling problem for a microgrid-powered MEC network. We derive the solution by applying a multi-agent deep reinforcement learning (MADRL)-based advantage actor-critic (A3C) algorithm with shared neural networks.
  arXiv  Detail & Related papers  (2020-02-21T02:14:38Z)
• Multi-Agent Meta-Reinforcement Learning for Self-Powered and Sustainable Edge Computing Systems [87.4519172058185]
  An effective energy dispatch mechanism for self-powered wireless networks with edge computing capabilities is studied. A novel multi-agent meta-reinforcement learning (MAMRL) framework is proposed to solve the formulated problem. Experimental results show that the proposed MAMRL model can reduce up to 11% non-renewable energy usage and by 22.4% the energy cost.
  arXiv  Detail & Related papers  (2020-02-20T04:58:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.