HDDLGym: A Tool for Studying Multi-Agent Hierarchical Problems Defined in HDDL with OpenAI Gym

• URL: http://arxiv.org/abs/2505.22597v1
• Date: Wed, 28 May 2025 17:10:43 GMT
• Title: HDDLGym: A Tool for Studying Multi-Agent Hierarchical Problems Defined in HDDL with OpenAI Gym
• Authors: Ngoc La, Ruaridh Mon-Williams, Julie A. Shah,
• Abstract summary: Reinforcement learning (RL) methods have been widely tested using tools like OpenAI Gym.<n>There is a lack of a tool that enables seamless integration of hierarchical planning with RL.<n>We introduce HDDLGym, a Python-based tool that automatically generates OpenAI Gym environments from HDDL domains and problems.
• Score: 7.365229047837842
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In recent years, reinforcement learning (RL) methods have been widely tested using tools like OpenAI Gym, though many tasks in these environments could also benefit from hierarchical planning. However, there is a lack of a tool that enables seamless integration of hierarchical planning with RL. Hierarchical Domain Definition Language (HDDL), used in classical planning, introduces a structured approach well-suited for model-based RL to address this gap. To bridge this integration, we introduce HDDLGym, a Python-based tool that automatically generates OpenAI Gym environments from HDDL domains and problems. HDDLGym serves as a link between RL and hierarchical planning, supporting multi-agent scenarios and enabling collaborative planning among agents. This paper provides an overview of HDDLGym's design and implementation, highlighting the challenges and design choices involved in integrating HDDL with the Gym interface, and applying RL policies to support hierarchical planning. We also provide detailed instructions and demonstrations for using the HDDLGym framework, including how to work with existing HDDL domains and problems from International Planning Competitions, exemplified by the Transport domain. Additionally, we offer guidance on creating new HDDL domains for multi-agent scenarios and demonstrate the practical use of HDDLGym in the Overcooked domain. By leveraging the advantages of HDDL and Gym, HDDLGym aims to be a valuable tool for studying RL in hierarchical planning, particularly in multi-agent contexts.

Related papers

• Graph-Augmented Large Language Model Agents: Current Progress and Future Prospects [53.24831948221361]
  Graph-augmented LLM Agents (GLA) enhance structure, continuity, and coordination in complex agent systems.<n>This paper offers a timely and comprehensive overview of recent advances and highlights key directions for future work.<n>We hope this paper can serve as a roadmap for future research on GLA and foster a deeper understanding of the role of graphs in GLA agent systems.
  arXiv  Detail & Related papers  (2025-07-29T00:27:12Z)
• HDLxGraph: Bridging Large Language Models and HDL Repositories via HDL Graph Databases [57.51078142561683]
  Large Language Models (LLMs) have demonstrated their potential in hardware design tasks.<n>Yet, their performance in real-world, repository-level HDL projects with thousands or even tens of thousands of code lines is hindered.<n>We propose HDLxGraph, a novel framework that integrates Graph Retrieval Augmented Generation (Graph RAG) with LLMs.
  arXiv  Detail & Related papers  (2025-05-21T16:14:10Z)
• Planning with Vision-Language Models and a Use Case in Robot-Assisted Teaching [0.9217021281095907]
  This paper introduces Image2PDDL, a novel framework that leverages Vision-Language Models (VLMs) to automatically convert images of initial states and descriptions of goal states into PDDL problems.<n>We evaluate the framework on various domains, including standard planning domains like blocksworld and sliding tile puzzles, using datasets with multiple difficulty levels.<n>We will discuss a potential use case in robot-assisted teaching of students with Autism Spectrum Disorder.
  arXiv  Detail & Related papers  (2025-01-29T14:04:54Z)
• Towards Human-Guided, Data-Centric LLM Co-Pilots [53.35493881390917]
  CliMB-DC is a human-guided, data-centric framework for machine learning co-pilots.<n>It combines advanced data-centric tools with LLM-driven reasoning to enable robust, context-aware data processing.<n>We show how CliMB-DC can transform uncurated datasets into ML-ready formats.
  arXiv  Detail & Related papers  (2025-01-17T17:51:22Z)
• LLAVIDAL: A Large LAnguage VIsion Model for Daily Activities of Living [15.062325300795257]
  Current Large Language Models (LLVMs) trained on web perform well in general understanding but struggle with fine-temporal details.<n>This limitation stems from a lack of specialized ADL video instruction-tuning and insufficient modality integration.<n>We propose a semi-grained framework for curating ADL datasets, creating ADL-X, a multiview, multimodal RGBS instruction-tuning dataset.<n>We also introduce LLAVIDAL, an LLVM integrating videos, 3D skeletons, and HOIs to model ADL's complex relationships.
  arXiv  Detail & Related papers  (2024-06-13T17:59:05Z)
• Real-World Planning with PDDL+ and Beyond [55.73913765642435]
  We present Nyx, a novel PDDL+ planner built to emphasize lightness, simplicity, and, most importantly, adaptability. Nyx can be tailored to virtually any potential real-world application requiring some form of AI Planning, paving the way for wider adoption of planning methods for solving real-world problems.
  arXiv  Detail & Related papers  (2024-02-19T07:35:49Z)
• Serving Deep Learning Model in Relational Databases [70.53282490832189]
  Serving deep learning (DL) models on relational data has become a critical requirement across diverse commercial and scientific domains. We highlight three pivotal paradigms: The state-of-the-art DL-centric architecture offloads DL computations to dedicated DL frameworks. The potential UDF-centric architecture encapsulates one or more tensor computations into User Defined Functions (UDFs) within the relational database management system (RDBMS)
  arXiv  Detail & Related papers  (2023-10-07T06:01:35Z)
• HDDL 2.1: Towards Defining a Formalism and a Semantics for Temporal HTN Planning [64.07762708909846]
  Real world applications need modelling rich and diverse automated planning problems. hierarchical task network (HTN) formalism does not allow to represent planning problems with numerical and temporal constraints. We propose to fill the gap between HDDL and these operational needs and to extend HDDL by taking inspiration from PDDL 2.1.
  arXiv  Detail & Related papers  (2023-06-12T18:21:23Z)
• HDDL 2.1: Towards Defining an HTN Formalism with Time [0.0]
  Real world applications of planning, like in industry and robotics, require modelling rich and diverse scenarios. Their resolution usually requires coordinated and concurrent action executions. In several cases, such planning problems are naturally decomposed in a hierarchical way and expressed by a Hierarchical Task Network formalism. This paper opens discussions on the semantics and the syntax needed to extend HDDL, and illustrate these needs with the modelling of an Earth Observing Satellite planning problem.
  arXiv  Detail & Related papers  (2022-06-03T21:22:19Z)
• PDDLGym: Gym Environments from PDDL Problems [13.630185187102413]
  We present PDDLGym, a framework that automatically constructs OpenAI Gym environments from PDDL domains and problems. Observations and actions in PDDLGym are relational, making the framework particularly well-suited for research in relational reinforcement learning and relational sequential decision-making.
  arXiv  Detail & Related papers  (2020-02-15T19:10:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.