Unsupervised Representation Learning in Partially Observable Atari Games

• URL: http://arxiv.org/abs/2303.07437v1
• Date: Mon, 13 Mar 2023 19:34:10 GMT
• Title: Unsupervised Representation Learning in Partially Observable Atari Games
• Authors: Li Meng, Morten Goodwin, Anis Yazidi, Paal Engelstad
• Abstract summary: State representation learning aims to capture latent factors of an environment. Contrastive methods have performed better than generative models in previous state representation learning research. In this article, we create an unsupervised state representation learning scheme for partially observable states.
• Score: 10.299850596045395
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: State representation learning aims to capture latent factors of an environment. Contrastive methods have performed better than generative models in previous state representation learning research. Although some researchers realize the connections between masked image modeling and contrastive representation learning, the effort is focused on using masks as an augmentation technique to represent the latent generative factors better. Partially observable environments in reinforcement learning have not yet been carefully studied using unsupervised state representation learning methods. In this article, we create an unsupervised state representation learning scheme for partially observable states. We conducted our experiment on a previous Atari 2600 framework designed to evaluate representation learning models. A contrastive method called Spatiotemporal DeepInfomax (ST-DIM) has shown state-of-the-art performance on this benchmark but remains inferior to its supervised counterpart. Our approach improves ST-DIM when the environment is not fully observable and achieves higher F1 scores and accuracy scores than the supervised learning counterpart. The mean accuracy score averaged over categories of our approach is ~66%, compared to ~38% of supervised learning. The mean F1 score is ~64% to ~33%.

Related papers

• HomE: Homography-Equivariant Video Representation Learning [62.89516761473129]
  We propose a novel method for representation learning of multi-view videos. Our method learns an implicit mapping between different views, culminating in a representation space that maintains the homography relationship between neighboring views. On action classification, our method obtains 96.4% 3-fold accuracy on the UCF101 dataset, better than most state-of-the-art self-supervised learning methods.
  arXiv  Detail & Related papers  (2023-06-02T15:37:43Z)
• Few-Shot Classification with Contrastive Learning [10.236150550121163]
  We propose a novel contrastive learning-based framework that seamlessly integrates contrastive learning into both stages. In the meta-training stage, we propose a cross-view episodic training mechanism to perform the nearest centroid classification on two different views of the same episode. These two strategies force the model to overcome the bias between views and promote the transferability of representations.
  arXiv  Detail & Related papers  (2022-09-17T02:39:09Z)
• Value-Consistent Representation Learning for Data-Efficient Reinforcement Learning [105.70602423944148]
  We propose a novel method, called value-consistent representation learning (VCR), to learn representations that are directly related to decision-making. Instead of aligning this imagined state with a real state returned by the environment, VCR applies a $Q$-value head on both states and obtains two distributions of action values. It has been demonstrated that our methods achieve new state-of-the-art performance for search-free RL algorithms.
  arXiv  Detail & Related papers  (2022-06-25T03:02:25Z)
• Weakly Supervised Contrastive Learning [68.47096022526927]
  We introduce a weakly supervised contrastive learning framework (WCL) to tackle this issue. WCL achieves 65% and 72% ImageNet Top-1 Accuracy using ResNet50, which is even higher than SimCLRv2 with ResNet101.
  arXiv  Detail & Related papers  (2021-10-10T12:03:52Z)
• Revisiting Contrastive Methods for Unsupervised Learning of Visual Representations [78.12377360145078]
  Contrastive self-supervised learning has outperformed supervised pretraining on many downstream tasks like segmentation and object detection. In this paper, we first study how biases in the dataset affect existing methods. We show that current contrastive approaches work surprisingly well across: (i) object- versus scene-centric, (ii) uniform versus long-tailed and (iii) general versus domain-specific datasets.
  arXiv  Detail & Related papers  (2021-06-10T17:59:13Z)
• A Large-Scale Study on Unsupervised Spatiotemporal Representation Learning [60.720251418816815]
  We present a large-scale study on unsupervised representation learning from videos. Our objective encourages temporally-persistent features in the same video. We find that encouraging long-spanned persistency can be effective even if the timespan is 60 seconds.
  arXiv  Detail & Related papers  (2021-04-29T17:59:53Z)
• What Makes for Good Views for Contrastive Learning? [90.49736973404046]
  We argue that we should reduce the mutual information (MI) between views while keeping task-relevant information intact. We devise unsupervised and semi-supervised frameworks that learn effective views by aiming to reduce their MI. As a by-product, we achieve a new state-of-the-art accuracy on unsupervised pre-training for ImageNet classification.
  arXiv  Detail & Related papers  (2020-05-20T17:59:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.