Unified State Representation Learning under Data Augmentation
- URL: http://arxiv.org/abs/2209.05302v1
- Date: Mon, 12 Sep 2022 15:10:28 GMT
- Title: Unified State Representation Learning under Data Augmentation
- Authors: Taylor Hearn, Sravan Jayanthi, Sehoon Ha
- Abstract summary: Generalization of reinforcement learning agents is critical to success in the real world.
We propose USRA: Unified State Representation Learning under Data Augmentation.
We find that USRA achieves higher sample efficiency and 14.3% better domain adaptation performance compared to the best baseline results.
- Score: 8.904143080467348
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The capacity for rapid domain adaptation is important to increasing the
applicability of reinforcement learning (RL) to real world problems.
Generalization of RL agents is critical to success in the real world, yet
zero-shot policy transfer is a challenging problem since even minor visual
changes could make the trained agent completely fail in the new task. We
propose USRA: Unified State Representation Learning under Data Augmentation, a
representation learning framework that learns a latent unified state
representation by performing data augmentations on its observations to improve
its ability to generalize to unseen target domains. We showcase the success of
our approach on the DeepMind Control Generalization Benchmark for the Walker
environment and find that USRA achieves higher sample efficiency and 14.3%
better domain adaptation performance compared to the best baseline results.
Related papers
- STAT: Towards Generalizable Temporal Action Localization [56.634561073746056]
Weakly-supervised temporal action localization (WTAL) aims to recognize and localize action instances with only video-level labels.
Existing methods suffer from severe performance degradation when transferring to different distributions.
We propose GTAL, which focuses on improving the generalizability of action localization methods.
arXiv Detail & Related papers (2024-04-20T07:56:21Z) - Invariance is Key to Generalization: Examining the Role of
Representation in Sim-to-Real Transfer for Visual Navigation [35.01394611106655]
Key to generalization is representations that are rich enough to capture all task-relevant information.
We experimentally study such a representation for visual navigation.
We show that our representation reduces the A-distance between the training and test domains.
arXiv Detail & Related papers (2023-10-23T15:15:19Z) - Improving Generalization of Alignment with Human Preferences through
Group Invariant Learning [56.19242260613749]
Reinforcement Learning from Human Feedback (RLHF) enables the generation of responses more aligned with human preferences.
Previous work shows that Reinforcement Learning (RL) often exploits shortcuts to attain high rewards and overlooks challenging samples.
We propose a novel approach that can learn a consistent policy via RL across various data groups or domains.
arXiv Detail & Related papers (2023-10-18T13:54:15Z) - CROP: Towards Distributional-Shift Robust Reinforcement Learning using
Compact Reshaped Observation Processing [8.569762036154799]
Current approaches for generalization apply data augmentation techniques to increase the diversity of training data.
Crafting a suitable observation, only containing crucial information, has been shown to be a challenging task itself.
We propose Compact Reshaped Observation Processing (CROP) to reduce the state information used for policy optimization.
arXiv Detail & Related papers (2023-04-26T15:19:02Z) - Human-Timescale Adaptation in an Open-Ended Task Space [56.55530165036327]
We show that training an RL agent at scale leads to a general in-context learning algorithm that can adapt to open-ended novel embodied 3D problems as quickly as humans.
Our results lay the foundation for increasingly general and adaptive RL agents that perform well across ever-larger open-ended domains.
arXiv Detail & Related papers (2023-01-18T15:39:21Z) - Weakly Supervised Disentangled Representation for Goal-conditioned
Reinforcement Learning [15.698612710580447]
We propose a skill learning framework DR-GRL that aims to improve the sample efficiency and policy generalization.
In a weakly supervised manner, we propose a Spatial Transform AutoEncoder (STAE) to learn an interpretable and controllable representation.
We empirically demonstrate that DR-GRL significantly outperforms the previous methods in sample efficiency and policy generalization.
arXiv Detail & Related papers (2022-02-28T09:05:14Z) - Unsupervised Domain Generalization for Person Re-identification: A
Domain-specific Adaptive Framework [50.88463458896428]
Domain generalization (DG) has attracted much attention in person re-identification (ReID) recently.
Existing methods usually need the source domains to be labeled, which could be a significant burden for practical ReID tasks.
We propose a simple and efficient domain-specific adaptive framework, and realize it with an adaptive normalization module.
arXiv Detail & Related papers (2021-11-30T02:35:51Z) - Exploratory State Representation Learning [63.942632088208505]
We propose a new approach called XSRL (eXploratory State Representation Learning) to solve the problems of exploration and SRL in parallel.
On one hand, it jointly learns compact state representations and a state transition estimator which is used to remove unexploitable information from the representations.
On the other hand, it continuously trains an inverse model, and adds to the prediction error of this model a $k$-step learning progress bonus to form the objective of a discovery policy.
arXiv Detail & Related papers (2021-09-28T10:11:07Z) - Generalization of Reinforcement Learning with Policy-Aware Adversarial
Data Augmentation [32.70482982044965]
We propose a novel policy-aware adversarial data augmentation method to augment the standard policy learning method with automatically generated trajectory data.
We conduct experiments on a number of RL tasks to investigate the generalization performance of the proposed method.
The results show our method can generalize well with limited training diversity, and achieve the state-of-the-art generalization test performance.
arXiv Detail & Related papers (2021-06-29T17:21:59Z) - Variational Empowerment as Representation Learning for Goal-Based
Reinforcement Learning [114.07623388322048]
We discuss how the standard goal-conditioned RL (GCRL) is encapsulated by the objective variational empowerment.
Our work lays a novel foundation from which to evaluate, analyze, and develop representation learning techniques in goal-based RL.
arXiv Detail & Related papers (2021-06-02T18:12:26Z) - Domain Adaptation In Reinforcement Learning Via Latent Unified State
Representation [1.435381256004719]
We propose a two-stage RL agent that first learns a latent unified state representation (LUSR) which is consistent across multiple domains in the first stage, and then do RL training in one source domain based on LUSR in the second stage.
Cross-domain consistency of LUSR allows the policy acquired from the source domain to generalize to other target domains without extra training.
Our results show that this approach can achieve state-of-the-art domain adaptation performance in related RL tasks and outperforms prior approaches based on latent-representation based RL and image-to-image translation.
arXiv Detail & Related papers (2021-02-10T19:38:14Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.