Unsupervised Representation Learning in Deep Reinforcement Learning: A Review

• URL: http://arxiv.org/abs/2208.14226v3
• Date: Wed, 1 May 2024 13:00:39 GMT
• Title: Unsupervised Representation Learning in Deep Reinforcement Learning: A Review
• Authors: Nicolò Botteghi, Mannes Poel, Christoph Brune,
• Abstract summary: This review addresses the problem of learning abstract representations of the measurement data in the context of Deep Reinforcement Learning (DRL) This review provides a comprehensive and complete overview of unsupervised representation learning in DRL by describing the main Deep Learning tools used for learning representations of the world.
• Score: 1.2016264781280588
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This review addresses the problem of learning abstract representations of the measurement data in the context of Deep Reinforcement Learning (DRL). While the data are often ambiguous, high-dimensional, and complex to interpret, many dynamical systems can be effectively described by a low-dimensional set of state variables. Discovering these state variables from the data is a crucial aspect for (i) improving the data efficiency, robustness, and generalization of DRL methods, (ii) tackling the curse of dimensionality, and (iii) bringing interpretability and insights into black-box DRL. This review provides a comprehensive and complete overview of unsupervised representation learning in DRL by describing the main Deep Learning tools used for learning representations of the world, providing a systematic view of the method and principles, summarizing applications, benchmarks and evaluation strategies, and discussing open challenges and future directions.

Related papers

• A Survey on Explainable Deep Reinforcement Learning [18.869827229746697]
  Deep Reinforcement Learning (DRL) has achieved remarkable success in sequential decision-making tasks across diverse domains. Its reliance on black-box neural architectures hinders interpretability, trust, and deployment in high-stakes applications. Explainable Deep Reinforcement Learning (XRL) addresses these challenges by enhancing transparency through feature-level, state-level, dataset-level, and model-level explanation techniques.
  arXiv  Detail & Related papers  (2025-02-08T05:30:31Z)
• Deep Learning for Video Anomaly Detection: A Review [52.74513211976795]
  Video anomaly detection (VAD) aims to discover behaviors or events deviating from the normality in videos. In the era of deep learning, a great variety of deep learning based methods are constantly emerging for the VAD task. This review covers the spectrum of five different categories, namely, semi-supervised, weakly supervised, fully supervised, unsupervised and open-set supervised VAD.
  arXiv  Detail & Related papers  (2024-09-09T07:31:16Z)
• Disentangled Representation Learning [46.51815065323667]
  Disentangled Representation Learning (DRL) aims to learn a model capable of identifying and disentangling the underlying factors hidden in the observable data in representation form. We comprehensively investigate DRL from various aspects including motivations, definitions, methodologies, evaluations, applications, and model designs.
  arXiv  Detail & Related papers  (2022-11-21T18:14:38Z)
• A Comprehensive Survey of Data Augmentation in Visual Reinforcement Learning [53.35317176453194]
  Data augmentation (DA) has become a widely used technique in visual RL for acquiring sample-efficient and generalizable policies. We present a principled taxonomy of the existing augmentation techniques used in visual RL and conduct an in-depth discussion on how to better leverage augmented data. As the first comprehensive survey of DA in visual RL, this work is expected to offer valuable guidance to this emerging field.
  arXiv  Detail & Related papers  (2022-10-10T11:01:57Z)
• Semi-Supervised and Unsupervised Deep Visual Learning: A Survey [76.2650734930974]
  Semi-supervised learning and unsupervised learning offer promising paradigms to learn from an abundance of unlabeled visual data. We review the recent advanced deep learning algorithms on semi-supervised learning (SSL) and unsupervised learning (UL) for visual recognition from a unified perspective.
  arXiv  Detail & Related papers  (2022-08-24T04:26:21Z)
• Challenges and Opportunities in Offline Reinforcement Learning from Visual Observations [58.758928936316785]
  offline reinforcement learning from visual observations with continuous action spaces remains under-explored. We show that modifications to two popular vision-based online reinforcement learning algorithms suffice to outperform existing offline RL methods.
  arXiv  Detail & Related papers  (2022-06-09T22:08:47Z)
• Causal Reasoning Meets Visual Representation Learning: A Prospective Study [117.08431221482638]
  Lack of interpretability, robustness, and out-of-distribution generalization are becoming the challenges of the existing visual models. Inspired by the strong inference ability of human-level agents, recent years have witnessed great effort in developing causal reasoning paradigms. This paper aims to provide a comprehensive overview of this emerging field, attract attention, encourage discussions, bring to the forefront the urgency of developing novel causal reasoning methods.
  arXiv  Detail & Related papers  (2022-04-26T02:22:28Z)
• Accelerating Representation Learning with View-Consistent Dynamics in Data-Efficient Reinforcement Learning [12.485293708638292]
  We propose to accelerate state representation learning by enforcing view-consistency on the dynamics. We introduce a formalism of Multi-view Markov Decision Process (MMDP) that incorporates multiple views of the state. Following the structure of MMDP, our method, View-Consistent Dynamics (VCD), learns state representations by training a view-consistent dynamics model in the latent space.
  arXiv  Detail & Related papers  (2022-01-18T14:28:30Z)
• Which Mutual-Information Representation Learning Objectives are Sufficient for Control? [80.2534918595143]
  Mutual information provides an appealing formalism for learning representations of data. This paper formalizes the sufficiency of a state representation for learning and representing the optimal policy. Surprisingly, we find that two of these objectives can yield insufficient representations given mild and common assumptions on the structure of the MDP.
  arXiv  Detail & Related papers  (2021-06-14T10:12:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.