MARLEM: A Multi-Agent Reinforcement Learning Simulation Framework for Implicit Cooperation in Decentralized Local Energy Markets

• URL: http://arxiv.org/abs/2602.16063v1
• Date: Tue, 17 Feb 2026 22:22:45 GMT
• Title: MARLEM: A Multi-Agent Reinforcement Learning Simulation Framework for Implicit Cooperation in Decentralized Local Energy Markets
• Authors: Nelson Salazar-Pena, Alejandra Tabares, Andres Gonzalez-Mancera,
• Abstract summary: This paper introduces a novel, open-source MARL simulation framework for studying implicit cooperation in LEMs.<n>Our framework features a modular market platform with plug-and-play clearing mechanisms, physically constrained agent models, and a realistic grid network.<n>The main contribution is a novel method to foster implicit cooperation, where agents' observations and rewards are enhanced with system-level key performance indicators.
• Score: 41.99844472131922
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper introduces a novel, open-source MARL simulation framework for studying implicit cooperation in LEMs, modeled as a decentralized partially observable Markov decision process and implemented as a Gymnasium environment for MARL. Our framework features a modular market platform with plug-and-play clearing mechanisms, physically constrained agent models (including battery storage), a realistic grid network, and a comprehensive analytics suite to evaluate emergent coordination. The main contribution is a novel method to foster implicit cooperation, where agents' observations and rewards are enhanced with system-level key performance indicators to enable them to independently learn strategies that benefit the entire system and aim for collectively beneficial outcomes without explicit communication. Through representative case studies (available in a dedicated GitHub repository in https://github.com/salazarna/marlem, we show the framework's ability to analyze how different market configurations (such as varying storage deployment) impact system performance. This illustrates its potential to facilitate emergent coordination, improve market efficiency, and strengthen grid stability. The proposed simulation framework is a flexible, extensible, and reproducible tool for researchers and practitioners to design, test, and validate strategies for future intelligent, decentralized energy systems.

Related papers

• Achieving Equilibrium under Utility Heterogeneity: An Agent-Attention Framework for Multi-Agent Multi-Objective Reinforcement Learning [30.138706163658597]
  We propose an Agent-Attention Multi-Agent Multi-Objective Reinforcement Learning (AA-MAMORL) framework.<n>Our approach implicitly learns a joint belief over other agents' utility functions and their associated policies during training.<n>In execution, each agent independently selects actions based on local observations and its private utility function to approximate a BNE.
  arXiv  Detail & Related papers  (2025-11-12T03:06:21Z)
• A Framework for Scalable Heterogeneous Multi-Agent Adversarial Reinforcement Learning in IsaacLab [1.5749416770494706]
  Multi-Agent Reinforcement Learning (MARL) is central to robotic systems cooperating in dynamic environments.<n>We extend the IsaacLab framework to support scalable training of adversarial policies in high-fidelity physics simulations.
  arXiv  Detail & Related papers  (2025-09-26T03:16:48Z)
• Hide-and-Shill: A Reinforcement Learning Framework for Market Manipulation Detection in Symphony-a Decentralized Multi-Agent System [7.392937244789759]
  Decentralized finance (DeFi) has introduced a new era of permissionless financial innovation but also led to unprecedented market manipulation.<n>We propose a Multi-Agent Reinforcement Learning framework for decentralized manipulation detection, modeling the interaction between manipulators and detectors as a dynamic adversarial game.<n>This framework identifies suspicious patterns using delayed token price reactions as financial indicators.
  arXiv  Detail & Related papers  (2025-07-12T07:55:40Z)
• MLE-Dojo: Interactive Environments for Empowering LLM Agents in Machine Learning Engineering [57.156093929365255]
  Gym-style framework for systematically reinforcement learning, evaluating, and improving autonomous large language model (LLM) agents.<n>MLE-Dojo covers diverse, open-ended MLE tasks carefully curated to reflect realistic engineering scenarios.<n>Its fully executable environment supports comprehensive agent training via both supervised fine-tuning and reinforcement learning.
  arXiv  Detail & Related papers  (2025-05-12T17:35:43Z)
• Robo-taxi Fleet Coordination at Scale via Reinforcement Learning [21.266509380044912]
  This work introduces a novel decision-making framework that unites mathematical modeling with data-driven techniques.<n>We present the AMoD coordination problem through the lens of reinforcement learning and propose a graph network-based framework.<n>In particular, we present the AMoD coordination problem through the lens of reinforcement learning and propose a graph network-based framework.
  arXiv  Detail & Related papers  (2025-04-08T15:19:41Z)
• Cooperative Multi-Agent Planning with Adaptive Skill Synthesis [16.228784877899976]
  We present a novel multi-agent architecture that integrates vision-language models (VLMs) with a dynamic skill library and structured communication for decentralized closed-loop decision-making.<n>The skill library, bootstrapped from demonstrations, evolves via planner-guided tasks to enable adaptive strategies.<n>We demonstrate its strong performance against state-of-the-art MARL baselines across both symmetric and asymmetric scenarios.
  arXiv  Detail & Related papers  (2025-02-14T13:23:18Z)
• A Local Information Aggregation based Multi-Agent Reinforcement Learning for Robot Swarm Dynamic Task Allocation [4.144893164317513]
  We introduce a novel framework using a decentralized partially observable Markov decision process (Dec_POMDP)<n>At the core of our methodology is the Local Information Aggregation Multi-Agent Deep Deterministic Policy Gradient (LIA_MADDPG) algorithm.<n>Our empirical evaluations show that the LIA module can be seamlessly integrated into various CTDE-based MARL methods.
  arXiv  Detail & Related papers  (2024-11-29T07:53:05Z)
• Efficient Model-based Multi-agent Reinforcement Learning via Optimistic Equilibrium Computation [93.52573037053449]
  H-MARL (Hallucinated Multi-Agent Reinforcement Learning) learns successful equilibrium policies after a few interactions with the environment. We demonstrate our approach experimentally on an autonomous driving simulation benchmark.
  arXiv  Detail & Related papers  (2022-03-14T17:24:03Z)
• Cooperative Policy Learning with Pre-trained Heterogeneous Observation Representations [51.8796674904734]
  We propose a new cooperative learning framework with pre-trained heterogeneous observation representations. We employ an encoder-decoder based graph attention to learn the intricate interactions and heterogeneous representations.
  arXiv  Detail & Related papers  (2020-12-24T04:52:29Z)
• Edge-assisted Democratized Learning Towards Federated Analytics [67.44078999945722]
  We show the hierarchical learning structure of the proposed edge-assisted democratized learning mechanism, namely Edge-DemLearn. We also validate Edge-DemLearn as a flexible model training mechanism to build a distributed control and aggregation methodology in regions.
  arXiv  Detail & Related papers  (2020-12-01T11:46:03Z)
• MARS-Gym: A Gym framework to model, train, and evaluate Recommender Systems for Marketplaces [51.123916699062384]
  MARS-Gym is an open-source framework to build and evaluate Reinforcement Learning agents for recommendations in marketplaces. We provide the implementation of a diverse set of baseline agents, with a metrics-driven analysis of them in the Trivago marketplace dataset. We expect to bridge the gap between academic research and production systems, as well as to facilitate the design of new algorithms and applications.
  arXiv  Detail & Related papers  (2020-09-30T16:39:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.