REPTILE: A Proactive Real-Time Deep Reinforcement Learning Self-adaptive Framework

• URL: http://arxiv.org/abs/2203.14686v1
• Date: Mon, 28 Mar 2022 12:38:08 GMT
• Title: REPTILE: A Proactive Real-Time Deep Reinforcement Learning Self-adaptive Framework
• Authors: Flavio Corradini, Miichele Loreti, Marco Piangerelli and Giacomo Rocchetti
• Abstract summary: A general framework is proposed to support the development of software systems that are able to adapt their behaviour according to the operating environment changes. The proposed approach, named REPTILE, works in a complete proactive manner and relies on Deep Reinforcement Learning-based agents to react to events. In our framework, two types of novelties are taken into account: those related to the context/environment and those related to the physical architecture itself. The framework, predicting those novelties before their occurrence, extracts time-changing models of the environment and uses a suitable Markov Decision Process to deal with the real-time setting.
• Score: 0.6335848702857039
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this work a general framework is proposed to support the development of software systems that are able to adapt their behaviour according to the operating environment changes. The proposed approach, named REPTILE, works in a complete proactive manner and relies on Deep Reinforcement Learning-based agents to react to events, referred as novelties, that can affect the expected behaviour of the system. In our framework, two types of novelties are taken into account: those related to the context/environment and those related to the physical architecture itself. The framework, predicting those novelties before their occurrence, extracts time-changing models of the environment and uses a suitable Markov Decision Process to deal with the real-time setting. Moreover, the architecture of our RL agent evolves based on the possible actions that can be taken.

Related papers

• Active Causal Structure Learning with Latent Variables: Towards Learning to Detour in Autonomous Robots [49.1574468325115]
  Artificial General Intelligence (AGI) Agents and Robots must be able to cope with everchanging environments and tasks. We claim that active causal structure learning with latent variables (ACSLWL) is a necessary component to build AGI agents and robots.
  arXiv  Detail & Related papers  (2024-10-28T10:21:26Z)
• PRACT: Optimizing Principled Reasoning and Acting of LLM Agent [96.10771520261596]
  We introduce the Principled Reasoning and Acting (PRAct) framework, a novel method for learning and enforcing action principles from trajectory data. We propose a new optimization framework, Reflective Principle Optimization (RPO), to adapt action principles to specific task requirements. Experimental results across four environments demonstrate that the PRAct agent, leveraging the RPO framework, effectively learns and applies action principles to enhance performance.
  arXiv  Detail & Related papers  (2024-10-24T08:21:51Z)
• Demonstrating the Continual Learning Capabilities and Practical Application of Discrete-Time Active Inference [0.0]
  Active inference is a mathematical framework for understanding how agents interact with their environments. In this paper, we present a continual learning framework for agents operating in discrete time environments. We demonstrate the agent's ability to relearn and refine its models efficiently, making it suitable for complex domains like finance and healthcare.
  arXiv  Detail & Related papers  (2024-09-30T21:18:46Z)
• Hierarchical Reinforcement Learning for Temporal Abstraction of Listwise Recommendation [51.06031200728449]
  We propose a novel framework called mccHRL to provide different levels of temporal abstraction on listwise recommendation. Within the hierarchical framework, the high-level agent studies the evolution of user perception, while the low-level agent produces the item selection policy. Results observe significant performance improvement by our method, compared with several well-known baselines.
  arXiv  Detail & Related papers  (2024-09-11T17:01:06Z)
• I Know How: Combining Prior Policies to Solve New Tasks [17.214443593424498]
  Multi-Task Reinforcement Learning aims at developing agents that are able to continually evolve and adapt to new scenarios. Learning from scratch for each new task is not a viable or sustainable option. We propose a new framework, I Know How, which provides a common formalization.
  arXiv  Detail & Related papers  (2024-06-14T08:44:51Z)
• Entropy-Regularized Token-Level Policy Optimization for Language Agent Reinforcement [67.1393112206885]
  Large Language Models (LLMs) have shown promise as intelligent agents in interactive decision-making tasks. We introduce Entropy-Regularized Token-level Policy Optimization (ETPO), an entropy-augmented RL method tailored for optimizing LLMs at the token level. We assess the effectiveness of ETPO within a simulated environment that models data science code generation as a series of multi-step interactive tasks.
  arXiv  Detail & Related papers  (2024-02-09T07:45:26Z)
• HAZARD Challenge: Embodied Decision Making in Dynamically Changing Environments [93.94020724735199]
  HAZARD consists of three unexpected disaster scenarios, including fire, flood, and wind. This benchmark enables us to evaluate autonomous agents' decision-making capabilities across various pipelines.
  arXiv  Detail & Related papers  (2024-01-23T18:59:43Z)
• Build generally reusable agent-environment interaction models [28.577502598559988]
  This paper tackles the problem of how to pre-train a model and make it generally reusable backbones for downstream task learning. We propose a method that builds an agent-environment interaction model by learning domain invariant successor features from the agent's vast experiences covering various tasks, then discretize them into behavior prototypes. We provide preliminary results that show downstream task learning based on a pre-trained embodied set structure can handle unseen changes in task objectives, environmental dynamics and sensor modalities.
  arXiv  Detail & Related papers  (2022-11-13T07:33:14Z)
• Embracing AWKWARD! Real-time Adjustment of Reactive Planning Using Social Norms [2.610470075814367]
  AWKWARD agents can have their plans re-configured in real time to align with social role requirements. OperA and BOD can achieve real-time adjustment of agent plans for evolving social roles.
  arXiv  Detail & Related papers  (2022-04-22T15:02:08Z)
• NovGrid: A Flexible Grid World for Evaluating Agent Response to Novelty [8.705624336757461]
  We introduce NovGrid, a novelty generation framework built on MiniGrid. Along with the core NovGrid we provide exemplar novelties aligned with our ontology and instantiate them as novelty templates. We present a set of metrics built into our framework for the evaluation of novelty-adaptation-enabled machine-learning techniques.
  arXiv  Detail & Related papers  (2022-03-23T01:06:04Z)
• Fast and Slow Learning of Recurrent Independent Mechanisms [80.38910637873066]
  We propose a training framework in which the pieces of knowledge an agent needs and its reward function are stationary and can be re-used across tasks. An attention mechanism dynamically selects which modules can be adapted to the current task. We find that meta-learning the modular aspects of the proposed system greatly helps in achieving faster adaptation in a reinforcement learning setup.
  arXiv  Detail & Related papers  (2021-05-18T17:50:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.