Generalising Discrete Action Spaces with Conditional Action Trees
- URL: http://arxiv.org/abs/2104.07294v1
- Date: Thu, 15 Apr 2021 08:10:18 GMT
- Title: Generalising Discrete Action Spaces with Conditional Action Trees
- Authors: Christopher Bamford, Alvaro Ovalle
- Abstract summary: We introduce em Conditional Action Trees with two main objectives.
We show several proof-of-concept experiments ranging from environments with discrete action spaces to those with large action spaces commonly found in RTS-style games.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: There are relatively few conventions followed in reinforcement learning (RL)
environments to structure the action spaces. As a consequence the application
of RL algorithms to tasks with large action spaces with multiple components
require additional effort to adjust to different formats. In this paper we
introduce {\em Conditional Action Trees} with two main objectives: (1) as a
method of structuring action spaces in RL to generalise across several action
space specifications, and (2) to formalise a process to significantly reduce
the action space by decomposing it into multiple sub-spaces, favoring a
multi-staged decision making approach. We show several proof-of-concept
experiments validating our scheme, ranging from environments with basic
discrete action spaces to those with large combinatorial action spaces commonly
found in RTS-style games.
Related papers
- Grounding Multimodal Large Language Models in Actions [65.88208317380793]
We study how to best ground a MLLM into different embodiments and their associated action spaces.
For continuous actions, we show that a learned tokenization allows for sufficient modeling precision.
For discrete actions, we demonstrate that semantically aligning these actions with the native output token space of the MLLM leads to the strongest performance.
arXiv Detail & Related papers (2024-06-12T06:12:04Z) - In-Context Reinforcement Learning for Variable Action Spaces [46.29510499540938]
Headless-AD is capable of generalizing to discrete action spaces of variable size, semantic content and order.
We show that Headless-AD exhibits significant capability to generalize to action spaces it has never encountered.
arXiv Detail & Related papers (2023-12-20T16:58:55Z) - Concrete Subspace Learning based Interference Elimination for Multi-task
Model Fusion [86.6191592951269]
Merging models fine-tuned from common extensively pretrained large model but specialized for different tasks has been demonstrated as a cheap and scalable strategy to construct a multitask model that performs well across diverse tasks.
We propose the CONtinuous relaxation dis (Concrete) subspace learning method to identify a common lowdimensional subspace and utilize its shared information track interference problem without sacrificing performance.
arXiv Detail & Related papers (2023-12-11T07:24:54Z) - AI planning in the imagination: High-level planning on learned abstract
search spaces [68.75684174531962]
We propose a new method, called PiZero, that gives an agent the ability to plan in an abstract search space that the agent learns during training.
We evaluate our method on multiple domains, including the traveling salesman problem, Sokoban, 2048, the facility location problem, and Pacman.
arXiv Detail & Related papers (2023-08-16T22:47:16Z) - Dynamic Neighborhood Construction for Structured Large Discrete Action
Spaces [2.285821277711785]
Large discrete action spaces (LDAS) remain a central challenge in reinforcement learning.
Existing solution approaches can handle unstructured LDAS with up to a few million actions.
We propose Dynamic Neighborhood Construction (DNC), a novel exploitation paradigm for SLDAS.
arXiv Detail & Related papers (2023-05-31T14:26:14Z) - Adaptive Discretization using Voronoi Trees for Continuous-Action POMDPs [7.713622698801596]
We propose a new sampling-based online POMDP solver, called Adaptive Discretization using Voronoi Trees (ADVT)
ADVT uses Monte Carlo Tree Search in combination with an adaptive discretization of the action space as well as optimistic optimization.
Experiments on simulations of four types of benchmark problems indicate that ADVT outperforms and scales substantially better to high-dimensional continuous action spaces.
arXiv Detail & Related papers (2022-09-13T05:04:49Z) - Graph Convolutional Module for Temporal Action Localization in Videos [142.5947904572949]
We claim that the relations between action units play an important role in action localization.
A more powerful action detector should not only capture the local content of each action unit but also allow a wider field of view on the context related to it.
We propose a general graph convolutional module (GCM) that can be easily plugged into existing action localization methods.
arXiv Detail & Related papers (2021-12-01T06:36:59Z) - Reinforcement Learning in Factored Action Spaces using Tensor
Decompositions [92.05556163518999]
We propose a novel solution for Reinforcement Learning (RL) in large, factored action spaces using tensor decompositions.
We use cooperative multi-agent reinforcement learning scenario as the exemplary setting.
arXiv Detail & Related papers (2021-10-27T15:49:52Z) - HyAR: Addressing Discrete-Continuous Action Reinforcement Learning via
Hybrid Action Representation [30.621472051415857]
Most previous Reinforcement Learning (RL) works only demonstrate the success in controlling with either discrete or continuous action space.
We propose Hybrid Action Representation (HyAR) to learn a compact and decodable latent representation space for the original hybrid action space.
We evaluate HyAR in a variety of environments with discrete-continuous action space.
arXiv Detail & Related papers (2021-09-12T11:26:27Z) - Exact Reduction of Huge Action Spaces in General Reinforcement Learning [28.19950790106291]
We show how action-binarization in the non-MDP case can significantly improve Extreme State Aggregation (ESA) bounds.
We provide an upper bound on the number of states of this binarized ESA that is logarithmic in the original action-space size, a double-exponential improvement.
arXiv Detail & Related papers (2020-12-18T12:45:03Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.