Case-based Reasoning for Better Generalization in Text-Adventure Games
- URL: http://arxiv.org/abs/2110.08470v1
- Date: Sat, 16 Oct 2021 04:51:34 GMT
- Title: Case-based Reasoning for Better Generalization in Text-Adventure Games
- Authors: Mattia Atzeni, Shehzaad Dhuliawala, Keerthiram Murugesan, Mrinmaya
Sachan
- Abstract summary: We propose a general method inspired by case-based reasoning to train agents and generalize out of the training distribution.
Our experiments show that the proposed approach consistently improves existing methods, obtains good out-of-distribution generalization, and achieves new state-of-the-art results on widely used environments.
- Score: 15.652823459179048
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Text-based games (TBG) have emerged as promising environments for driving
research in grounded language understanding and studying problems like
generalization and sample efficiency. Several deep reinforcement learning (RL)
methods with varying architectures and learning schemes have been proposed for
TBGs. However, these methods fail to generalize efficiently, especially under
distributional shifts. In a departure from deep RL approaches, in this paper,
we propose a general method inspired by case-based reasoning to train agents
and generalize out of the training distribution. The case-based reasoner
collects instances of positive experiences from the agent's interaction with
the world in the past and later reuses the collected experiences to act
efficiently. The method can be applied in conjunction with any existing
on-policy neural agent in the literature for TBGs. Our experiments show that
the proposed approach consistently improves existing methods, obtains good
out-of-distribution generalization, and achieves new state-of-the-art results
on widely used environments.
Related papers
- NormAUG: Normalization-guided Augmentation for Domain Generalization [60.159546669021346]
We propose a simple yet effective method called NormAUG (Normalization-guided Augmentation) for deep learning.
Our method introduces diverse information at the feature level and improves the generalization of the main path.
In the test stage, we leverage an ensemble strategy to combine the predictions from the auxiliary path of our model, further boosting performance.
arXiv Detail & Related papers (2023-07-25T13:35:45Z) - On the Importance of Exploration for Generalization in Reinforcement
Learning [89.63074327328765]
We propose EDE: Exploration via Distributional Ensemble, a method that encourages exploration of states with high uncertainty.
Our algorithm is the first value-based approach to achieve state-of-the-art on both Procgen and Crafter.
arXiv Detail & Related papers (2023-06-08T18:07:02Z) - Retrieval Augmentation for Commonsense Reasoning: A Unified Approach [64.63071051375289]
We propose a unified framework of retrieval-augmented commonsense reasoning (called RACo)
Our proposed RACo can significantly outperform other knowledge-enhanced method counterparts.
arXiv Detail & Related papers (2022-10-23T23:49:08Z) - A Game-Theoretic Perspective of Generalization in Reinforcement Learning [9.402272029807316]
Generalization in reinforcement learning (RL) is of importance for real deployment of RL algorithms.
We propose a game-theoretic framework for the generalization in reinforcement learning, named GiRL.
arXiv Detail & Related papers (2022-08-07T06:17:15Z) - A Survey of Generalisation in Deep Reinforcement Learning [18.098133342169646]
Generalisation in deep Reinforcement Learning aims to produce RL algorithms whose policies generalise well to novel unseen situations at deployment time.
Tackling this is vital if we are to deploy reinforcement learning algorithms in real world scenarios.
This survey is an overview of this nascent field.
arXiv Detail & Related papers (2021-11-18T16:53:02Z) - Generalization of Reinforcement Learning with Policy-Aware Adversarial
Data Augmentation [32.70482982044965]
We propose a novel policy-aware adversarial data augmentation method to augment the standard policy learning method with automatically generated trajectory data.
We conduct experiments on a number of RL tasks to investigate the generalization performance of the proposed method.
The results show our method can generalize well with limited training diversity, and achieve the state-of-the-art generalization test performance.
arXiv Detail & Related papers (2021-06-29T17:21:59Z) - Off-Policy Imitation Learning from Observations [78.30794935265425]
Learning from Observations (LfO) is a practical reinforcement learning scenario from which many applications can benefit.
We propose a sample-efficient LfO approach that enables off-policy optimization in a principled manner.
Our approach is comparable with state-of-the-art locomotion in terms of both sample-efficiency and performance.
arXiv Detail & Related papers (2021-02-25T21:33:47Z) - Dynamics Generalization via Information Bottleneck in Deep Reinforcement
Learning [90.93035276307239]
We propose an information theoretic regularization objective and an annealing-based optimization method to achieve better generalization ability in RL agents.
We demonstrate the extreme generalization benefits of our approach in different domains ranging from maze navigation to robotic tasks.
This work provides a principled way to improve generalization in RL by gradually removing information that is redundant for task-solving.
arXiv Detail & Related papers (2020-08-03T02:24:20Z) - Invariant Causal Prediction for Block MDPs [106.63346115341862]
Generalization across environments is critical to the successful application of reinforcement learning algorithms to real-world challenges.
We propose a method of invariant prediction to learn model-irrelevance state abstractions (MISA) that generalize to novel observations in the multi-environment setting.
arXiv Detail & Related papers (2020-03-12T21:03:01Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.