REPTILE: A Proactive Real-Time Deep Reinforcement Learning Self-adaptive
Framework
- URL: http://arxiv.org/abs/2203.14686v1
- Date: Mon, 28 Mar 2022 12:38:08 GMT
- Title: REPTILE: A Proactive Real-Time Deep Reinforcement Learning Self-adaptive
Framework
- Authors: Flavio Corradini, Miichele Loreti, Marco Piangerelli and Giacomo
Rocchetti
- Abstract summary: A general framework is proposed to support the development of software systems that are able to adapt their behaviour according to the operating environment changes.
The proposed approach, named REPTILE, works in a complete proactive manner and relies on Deep Reinforcement Learning-based agents to react to events.
In our framework, two types of novelties are taken into account: those related to the context/environment and those related to the physical architecture itself.
The framework, predicting those novelties before their occurrence, extracts time-changing models of the environment and uses a suitable Markov Decision Process to deal with the real-time setting.
- Score: 0.6335848702857039
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: In this work a general framework is proposed to support the development of
software systems that are able to adapt their behaviour according to the
operating environment changes. The proposed approach, named REPTILE, works in a
complete proactive manner and relies on Deep Reinforcement Learning-based
agents to react to events, referred as novelties, that can affect the expected
behaviour of the system. In our framework, two types of novelties are taken
into account: those related to the context/environment and those related to the
physical architecture itself. The framework, predicting those novelties before
their occurrence, extracts time-changing models of the environment and uses a
suitable Markov Decision Process to deal with the real-time setting. Moreover,
the architecture of our RL agent evolves based on the possible actions that can
be taken.
Related papers
- I Know How: Combining Prior Policies to Solve New Tasks [17.214443593424498]
Multi-Task Reinforcement Learning aims at developing agents that are able to continually evolve and adapt to new scenarios.
Learning from scratch for each new task is not a viable or sustainable option.
We propose a new framework, I Know How, which provides a common formalization.
arXiv Detail & Related papers (2024-06-14T08:44:51Z) - Jointly Training and Pruning CNNs via Learnable Agent Guidance and Alignment [69.33930972652594]
We propose a novel structural pruning approach to jointly learn the weights and structurally prune architectures of CNN models.
The core element of our method is a Reinforcement Learning (RL) agent whose actions determine the pruning ratios of the CNN model's layers.
We conduct the joint training and pruning by iteratively training the model's weights and the agent's policy.
arXiv Detail & Related papers (2024-03-28T15:22:29Z) - Entropy-Regularized Token-Level Policy Optimization for Language Agent Reinforcement [67.1393112206885]
Large Language Models (LLMs) have shown promise as intelligent agents in interactive decision-making tasks.
We introduce Entropy-Regularized Token-level Policy Optimization (ETPO), an entropy-augmented RL method tailored for optimizing LLMs at the token level.
We assess the effectiveness of ETPO within a simulated environment that models data science code generation as a series of multi-step interactive tasks.
arXiv Detail & Related papers (2024-02-09T07:45:26Z) - HAZARD Challenge: Embodied Decision Making in Dynamically Changing
Environments [93.94020724735199]
HAZARD consists of three unexpected disaster scenarios, including fire, flood, and wind.
This benchmark enables us to evaluate autonomous agents' decision-making capabilities across various pipelines.
arXiv Detail & Related papers (2024-01-23T18:59:43Z) - Build generally reusable agent-environment interaction models [28.577502598559988]
This paper tackles the problem of how to pre-train a model and make it generally reusable backbones for downstream task learning.
We propose a method that builds an agent-environment interaction model by learning domain invariant successor features from the agent's vast experiences covering various tasks, then discretize them into behavior prototypes.
We provide preliminary results that show downstream task learning based on a pre-trained embodied set structure can handle unseen changes in task objectives, environmental dynamics and sensor modalities.
arXiv Detail & Related papers (2022-11-13T07:33:14Z) - A Modular Framework for Reinforcement Learning Optimal Execution [68.8204255655161]
We develop a modular framework for the application of Reinforcement Learning to the problem of Optimal Trade Execution.
The framework is designed with flexibility in mind, in order to ease the implementation of different simulation setups.
arXiv Detail & Related papers (2022-08-11T09:40:42Z) - Pessimism meets VCG: Learning Dynamic Mechanism Design via Offline
Reinforcement Learning [114.36124979578896]
We design a dynamic mechanism using offline reinforcement learning algorithms.
Our algorithm is based on the pessimism principle and only requires a mild assumption on the coverage of the offline data set.
arXiv Detail & Related papers (2022-05-05T05:44:26Z) - Embracing AWKWARD! Real-time Adjustment of Reactive Planning Using
Social Norms [2.610470075814367]
AWKWARD agents can have their plans re-configured in real time to align with social role requirements.
OperA and BOD can achieve real-time adjustment of agent plans for evolving social roles.
arXiv Detail & Related papers (2022-04-22T15:02:08Z) - NovGrid: A Flexible Grid World for Evaluating Agent Response to Novelty [8.705624336757461]
We introduce NovGrid, a novelty generation framework built on MiniGrid.
Along with the core NovGrid we provide exemplar novelties aligned with our ontology and instantiate them as novelty templates.
We present a set of metrics built into our framework for the evaluation of novelty-adaptation-enabled machine-learning techniques.
arXiv Detail & Related papers (2022-03-23T01:06:04Z) - Fast and Slow Learning of Recurrent Independent Mechanisms [80.38910637873066]
We propose a training framework in which the pieces of knowledge an agent needs and its reward function are stationary and can be re-used across tasks.
An attention mechanism dynamically selects which modules can be adapted to the current task.
We find that meta-learning the modular aspects of the proposed system greatly helps in achieving faster adaptation in a reinforcement learning setup.
arXiv Detail & Related papers (2021-05-18T17:50:32Z) - Reinforcement Learning with Algorithms from Probabilistic Structure
Estimation [9.37335587960084]
Reinforcement learning algorithms aim to learn optimal decisions in unknown environments.
It is unknown from the outset whether or not the agent's actions will impact the environment.
It is often not possible to determine which RL algorithm is most fitting.
arXiv Detail & Related papers (2021-03-15T09:51:34Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.