CURL: Contrastive Unsupervised Representations for Reinforcement Learning

• URL: http://arxiv.org/abs/2004.04136v4
• Date: Mon, 21 Sep 2020 15:34:30 GMT
• Title: CURL: Contrastive Unsupervised Representations for Reinforcement Learning
• Authors: Aravind Srinivas, Michael Laskin, Pieter Abbeel
• Abstract summary: CURL extracts high-level features from raw pixels using contrastive learning. On the DeepMind Control Suite, CURL is the first image-based algorithm to nearly match the sample-efficiency of methods that use state-based features.
• Score: 93.57637441080603
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present CURL: Contrastive Unsupervised Representations for Reinforcement Learning. CURL extracts high-level features from raw pixels using contrastive learning and performs off-policy control on top of the extracted features. CURL outperforms prior pixel-based methods, both model-based and model-free, on complex tasks in the DeepMind Control Suite and Atari Games showing 1.9x and 1.2x performance gains at the 100K environment and interaction steps benchmarks respectively. On the DeepMind Control Suite, CURL is the first image-based algorithm to nearly match the sample-efficiency of methods that use state-based features. Our code is open-sourced and available at https://github.com/MishaLaskin/curl.

Related papers

• CricaVPR: Cross-image Correlation-aware Representation Learning for Visual Place Recognition [73.51329037954866]
  We propose a robust global representation method with cross-image correlation awareness for visual place recognition. Our method uses the attention mechanism to correlate multiple images within a batch. Our method outperforms state-of-the-art methods by a large margin with significantly less training time.
  arXiv  Detail & Related papers  (2024-02-29T15:05:11Z)
• Efficient Model-Based Concave Utility Reinforcement Learning through Greedy Mirror Descent [0.0]
  Concave Utility Reinforcement Learning problem invalidates classical Bellman equations. We introduce MD-CURL, a new algorithm for CURL in a finite horizon Markov decision process. We present Greedy MD-CURL, a new method adapting MD-CURL to an online, episode-based setting.
  arXiv  Detail & Related papers  (2023-11-30T08:32:50Z)
• DetCo: Unsupervised Contrastive Learning for Object Detection [64.22416613061888]
  Unsupervised contrastive learning achieves great success in learning image representations with CNN. We present a novel contrastive learning approach, named DetCo, which fully explores the contrasts between global image and local image patches. DetCo consistently outperforms supervised method by 1.6/1.2/1.0 AP on Mask RCNN-C4/FPN/RetinaNet with 1x schedule.
  arXiv  Detail & Related papers  (2021-02-09T12:47:20Z)
• Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learning [60.75687261314962]
  We introduce pixel-level pretext tasks for learning dense feature representations. A pixel-to-propagation consistency task produces better results than state-of-the-art approaches. Results demonstrate the strong potential of defining pretext tasks at the pixel level.
  arXiv  Detail & Related papers  (2020-11-19T18:59:45Z)
• Dense Contrastive Learning for Self-Supervised Visual Pre-Training [102.15325936477362]
  We present dense contrastive learning, which implements self-supervised learning by optimizing a pairwise contrastive (dis)similarity loss at the pixel level between two views of input images. Compared to the baseline method MoCo-v2, our method introduces negligible computation overhead (only 1% slower)
  arXiv  Detail & Related papers  (2020-11-18T08:42:32Z)
• Masked Contrastive Representation Learning for Reinforcement Learning [202.8261654227565]
  CURL, which uses contrastive learning to extract high-level features from raw pixels of individual video frames, is an efficient algorithm. We propose a new algorithm, masked contrastive representation learning for RL, that takes the correlation among consecutive inputs into consideration. Our method achieves consistent improvements over CURL on $14$ out of $16$ environments from DMControl suite and $21$ out of $26$ environments from Atari 2600 Games.
  arXiv  Detail & Related papers  (2020-10-15T02:00:10Z)
• Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels [37.726433732939114]
  We propose a simple data augmentation technique that can be applied to standard model-free reinforcement learning algorithms. We leverage input perturbations commonly used in computer vision tasks to regularize the value function. Our approach can be combined with any model-free reinforcement learning algorithm, requiring only minor modifications.
  arXiv  Detail & Related papers  (2020-04-28T16:48:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.