RAPid-Learn: A Framework for Learning to Recover for Handling Novelties in Open-World Environments

• URL: http://arxiv.org/abs/2206.12493v1
• Date: Fri, 24 Jun 2022 21:40:10 GMT
• Title: RAPid-Learn: A Framework for Learning to Recover for Handling Novelties in Open-World Environments
• Authors: Shivam Goel, Yash Shukla, Vasanth Sarathy, Matthias Scheutz, Jivko Sinapov
• Abstract summary: RAPid-Learn is designed to formulate and solve modifications to a task's Markov Decision Process (MDPs) on-the-fly. It is capable of exploiting domain knowledge to learn any new dynamics caused by the environmental changes. We demonstrate its efficacy by introducing a wide variety of novelties in a gridworld environment inspired by Minecraft.
• Score: 17.73296831597868
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose RAPid-Learn: Learning to Recover and Plan Again, a hybrid planning and learning method, to tackle the problem of adapting to sudden and unexpected changes in an agent's environment (i.e., novelties). RAPid-Learn is designed to formulate and solve modifications to a task's Markov Decision Process (MDPs) on-the-fly and is capable of exploiting domain knowledge to learn any new dynamics caused by the environmental changes. It is capable of exploiting the domain knowledge to learn action executors which can be further used to resolve execution impasses, leading to a successful plan execution. This novelty information is reflected in its updated domain model. We demonstrate its efficacy by introducing a wide variety of novelties in a gridworld environment inspired by Minecraft, and compare our algorithm with transfer learning baselines from the literature. Our method is (1) effective even in the presence of multiple novelties, (2) more sample efficient than transfer learning RL baselines, and (3) robust to incomplete model information, as opposed to pure symbolic planning approaches.

Related papers

• ReLIC: A Recipe for 64k Steps of In-Context Reinforcement Learning for Embodied AI [44.77897322913095]
  We present ReLIC, a new approach for in-context reinforcement learning for embodied agents. With ReLIC, agents are capable of adapting to new environments using 64,000 steps of in-context experience. We find that ReLIC is capable of few-shot imitation learning despite never being trained with expert demonstrations.
  arXiv  Detail & Related papers  (2024-10-03T17:58:11Z)
• Subspace Distillation for Continual Learning [27.22147868163214]
  We propose a knowledge distillation technique that takes into account the manifold structure of a neural network in learning novel tasks. We demonstrate that the modeling with subspaces provides several intriguing properties, including robustness to noise. Empirically, we observe that our proposed method outperforms various continual learning methods on several challenging datasets.
  arXiv  Detail & Related papers  (2023-07-31T05:59:09Z)
• Model-Based Reinforcement Learning with Multi-Task Offline Pretraining [59.82457030180094]
  We present a model-based RL method that learns to transfer potentially useful dynamics and action demonstrations from offline data to a novel task. The main idea is to use the world models not only as simulators for behavior learning but also as tools to measure the task relevance. We demonstrate the advantages of our approach compared with the state-of-the-art methods in Meta-World and DeepMind Control Suite.
  arXiv  Detail & Related papers  (2023-06-06T02:24:41Z)
• Domain Adapting Speech Emotion Recognition modals to real-world scenario with Deep Reinforcement Learning [5.40755576668989]
  Domain adaptation allows us to transfer knowledge learnt by a model across domains after a phase of training. We present a deep reinforcement learning-based strategy for adapting a pre-trained model to a newer domain.
  arXiv  Detail & Related papers  (2022-07-07T02:53:39Z)
• Self-Supervised Graph Neural Network for Multi-Source Domain Adaptation [51.21190751266442]
  Domain adaptation (DA) tries to tackle the scenarios when the test data does not fully follow the same distribution of the training data. By learning from large-scale unlabeled samples, self-supervised learning has now become a new trend in deep learning. We propose a novel textbfSelf-textbfSupervised textbfGraph Neural Network (SSG) to enable more effective inter-task information exchange and knowledge sharing.
  arXiv  Detail & Related papers  (2022-04-08T03:37:56Z)
• Creativity of AI: Hierarchical Planning Model Learning for Facilitating Deep Reinforcement Learning [19.470693909025798]
  We introduce a novel deep reinforcement learning framework with symbolic options. Our framework features a loop training procedure, which enables guiding the improvement of policy. We conduct experiments on two domains, Montezuma's Revenge and Office World, respectively.
  arXiv  Detail & Related papers  (2021-12-18T03:45:28Z)
• Procedure Planning in Instructional Videosvia Contextual Modeling and Model-based Policy Learning [114.1830997893756]
  This work focuses on learning a model to plan goal-directed actions in real-life videos. We propose novel algorithms to model human behaviors through Bayesian Inference and model-based Imitation Learning.
  arXiv  Detail & Related papers  (2021-10-05T01:06:53Z)
• Incremental Embedding Learning via Zero-Shot Translation [65.94349068508863]
  Current state-of-the-art incremental learning methods tackle catastrophic forgetting problem in traditional classification networks. We propose a novel class-incremental method for embedding network, named as zero-shot translation class-incremental method (ZSTCI) In addition, ZSTCI can easily be combined with existing regularization-based incremental learning methods to further improve performance of embedding networks.
  arXiv  Detail & Related papers  (2020-12-31T08:21:37Z)
• Meta-learning the Learning Trends Shared Across Tasks [123.10294801296926]
  Gradient-based meta-learning algorithms excel at quick adaptation to new tasks with limited data. Existing meta-learning approaches only depend on the current task information during the adaptation. We propose a 'Path-aware' model-agnostic meta-learning approach.
  arXiv  Detail & Related papers  (2020-10-19T08:06:47Z)
• Delta Schema Network in Model-based Reinforcement Learning [125.99533416395765]
  This work is devoted to unresolved problems of Artificial General Intelligence - the inefficiency of transfer learning. We are expanding the schema networks method which allows to extract the logical relationships between objects and actions from the environment data. We present algorithms for training a Delta Network (DSN), predicting future states of the environment and planning actions that will lead to positive reward.
  arXiv  Detail & Related papers  (2020-06-17T15:58:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.