Palm up: Playing in the Latent Manifold for Unsupervised Pretraining

• URL: http://arxiv.org/abs/2210.10913v1
• Date: Wed, 19 Oct 2022 22:26:12 GMT
• Title: Palm up: Playing in the Latent Manifold for Unsupervised Pretraining
• Authors: Hao Liu, Tom Zahavy, Volodymyr Mnih, Satinder Singh
• Abstract summary: We propose an algorithm that exhibits an exploratory behavior whilst it utilizes large diverse datasets. Our key idea is to leverage deep generative models that are pretrained on static datasets and introduce a dynamic model in the latent space. We then employ an unsupervised reinforcement learning algorithm to explore in this environment and perform unsupervised representation learning on the collected data.
• Score: 31.92145741769497
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large and diverse datasets have been the cornerstones of many impressive advancements in artificial intelligence. Intelligent creatures, however, learn by interacting with the environment, which changes the input sensory signals and the state of the environment. In this work, we aim to bring the best of both worlds and propose an algorithm that exhibits an exploratory behavior whilst it utilizes large diverse datasets. Our key idea is to leverage deep generative models that are pretrained on static datasets and introduce a dynamic model in the latent space. The transition dynamics simply mixes an action and a random sampled latent. It then applies an exponential moving average for temporal persistency, the resulting latent is decoded to image using pretrained generator. We then employ an unsupervised reinforcement learning algorithm to explore in this environment and perform unsupervised representation learning on the collected data. We further leverage the temporal information of this data to pair data points as a natural supervision for representation learning. Our experiments suggest that the learned representations can be successfully transferred to downstream tasks in both vision and reinforcement learning domains.

Related papers

• Reinforcement Learning Based Multi-modal Feature Fusion Network for Novel Class Discovery [47.28191501836041]
  In this paper, we employ a Reinforcement Learning framework to simulate the cognitive processes of humans. We also deploy a Member-to-Leader Multi-Agent framework to extract and fuse features from multi-modal information. We demonstrate the performance of our approach in both the 3D and 2D domains by employing the OS-MN40, OS-MN40-Miss, and Cifar10 datasets.
  arXiv  Detail & Related papers  (2023-08-26T07:55:32Z)
• ALP: Action-Aware Embodied Learning for Perception [60.64801970249279]
  We introduce Action-Aware Embodied Learning for Perception (ALP) ALP incorporates action information into representation learning through a combination of optimizing a reinforcement learning policy and an inverse dynamics prediction objective. We show that ALP outperforms existing baselines in several downstream perception tasks.
  arXiv  Detail & Related papers  (2023-06-16T21:51:04Z)
• Reinforcement Learning from Passive Data via Latent Intentions [86.4969514480008]
  We show that passive data can still be used to learn features that accelerate downstream RL. Our approach learns from passive data by modeling intentions. Our experiments demonstrate the ability to learn from many forms of passive data, including cross-embodiment video data and YouTube videos.
  arXiv  Detail & Related papers  (2023-04-10T17:59:05Z)
• Self-supervised Sequential Information Bottleneck for Robust Exploration in Deep Reinforcement Learning [28.75574762244266]
  In this work, we introduce the sequential information bottleneck objective for learning compressed and temporally coherent representations. For efficient exploration in noisy environments, we further construct intrinsic rewards that capture task-relevant state novelty.
  arXiv  Detail & Related papers  (2022-09-12T15:41:10Z)
• Stochastic Coherence Over Attention Trajectory For Continuous Learning In Video Streams [64.82800502603138]
  This paper proposes a novel neural-network-based approach to progressively and autonomously develop pixel-wise representations in a video stream. The proposed method is based on a human-like attention mechanism that allows the agent to learn by observing what is moving in the attended locations. Our experiments leverage 3D virtual environments and they show that the proposed agents can learn to distinguish objects just by observing the video stream.
  arXiv  Detail & Related papers  (2022-04-26T09:52:31Z)
• Compressed Predictive Information Coding [6.220929746808418]
  We develop a novel information-theoretic framework, Compressed Predictive Information Coding (CPIC), to extract useful representations from dynamic data. We derive variational bounds of the CPIC loss which induces the latent space to capture information that is maximally predictive. We demonstrate that CPIC is able to recover the latent space of noisy dynamical systems with low signal-to-noise ratios.
  arXiv  Detail & Related papers  (2022-03-03T22:47:58Z)
• Self-supervised Audiovisual Representation Learning for Remote Sensing Data [96.23611272637943]
  We propose a self-supervised approach for pre-training deep neural networks in remote sensing. By exploiting the correspondence between geo-tagged audio recordings and remote sensing, this is done in a completely label-free manner. We show that our approach outperforms existing pre-training strategies for remote sensing imagery.
  arXiv  Detail & Related papers  (2021-08-02T07:50:50Z)
• Curious Representation Learning for Embodied Intelligence [81.21764276106924]
  Self-supervised representation learning has achieved remarkable success in recent years. Yet to build truly intelligent agents, we must construct representation learning algorithms that can learn from environments. We propose a framework, curious representation learning, which jointly learns a reinforcement learning policy and a visual representation model.
  arXiv  Detail & Related papers  (2021-05-03T17:59:20Z)
• Laplacian Denoising Autoencoder [114.21219514831343]
  We propose to learn data representations with a novel type of denoising autoencoder. The noisy input data is generated by corrupting latent clean data in the gradient domain. Experiments on several visual benchmarks demonstrate that better representations can be learned with the proposed approach.
  arXiv  Detail & Related papers  (2020-03-30T16:52:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.