Skill Learning via Policy Diversity Yields Identifiable Representations for Reinforcement Learning

• URL: http://arxiv.org/abs/2507.14748v1
• Date: Sat, 19 Jul 2025 20:48:46 GMT
• Title: Skill Learning via Policy Diversity Yields Identifiable Representations for Reinforcement Learning
• Authors: Patrik Reizinger, Bálint Mucsányi, Siyuan Guo, Benjamin Eysenbach, Bernhard Schölkopf, Wieland Brendel,
• Abstract summary: Self-supervised feature learning (RL) often rely on information-theoretic principles, termed mutual information skill learning (MISL)<n>Our work investigates MISL through the lens of identifiable representation learning.<n>We prove that Contrastive Successor Features (CSF) can provably recover the environment's ground-truth features up to a linear transformation.
• Score: 67.90033766878962
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Self-supervised feature learning and pretraining methods in reinforcement learning (RL) often rely on information-theoretic principles, termed mutual information skill learning (MISL). These methods aim to learn a representation of the environment while also incentivizing exploration thereof. However, the role of the representation and mutual information parametrization in MISL is not yet well understood theoretically. Our work investigates MISL through the lens of identifiable representation learning by focusing on the Contrastive Successor Features (CSF) method. We prove that CSF can provably recover the environment's ground-truth features up to a linear transformation due to the inner product parametrization of the features and skill diversity in a discriminative sense. This first identifiability guarantee for representation learning in RL also helps explain the implications of different mutual information objectives and the downsides of entropy regularizers. We empirically validate our claims in MuJoCo and DeepMind Control and show how CSF provably recovers the ground-truth features both from states and pixels.

Related papers

• Identifiable Exchangeable Mechanisms for Causal Structure and Representation Learning [54.69189620971405]
  We provide a unified framework, termed Identifiable Exchangeable Mechanisms (IEM), for representation and structure learning.<n>IEM provides new insights that let us relax the necessary conditions for causal structure identification in exchangeable non-i.i.d. data.<n>We also demonstrate the existence of a duality condition in identifiable representation learning, leading to new identifiability results.
  arXiv  Detail & Related papers  (2024-06-20T13:30:25Z)
• A Review on Discriminative Self-supervised Learning Methods in Computer Vision [5.5547914920738]
  Self-supervised learning (SSL) has rapidly emerged as a transformative approach in computer vision.<n>This review presents a comprehensive analysis of discriminative SSL methods, which focus on learning representations by solving pretext tasks that do not require human labels.
  arXiv  Detail & Related papers  (2024-05-08T11:15:20Z)
• A Theoretical Analysis of Self-Supervised Learning for Vision Transformers [66.08606211686339]
  Masked autoencoders (MAE) and contrastive learning (CL) capture different types of representations.<n>We study the training dynamics of one-layer softmax-based vision transformers (ViTs) on both MAE and CL objectives.
  arXiv  Detail & Related papers  (2024-03-04T17:24:03Z)
• C-ICL: Contrastive In-context Learning for Information Extraction [54.39470114243744]
  c-ICL is a novel few-shot technique that leverages both correct and incorrect sample constructions to create in-context learning demonstrations. Our experiments on various datasets indicate that c-ICL outperforms previous few-shot in-context learning methods.
  arXiv  Detail & Related papers  (2024-02-17T11:28:08Z)
• Towards Uncovering How Large Language Model Works: An Explainability Perspective [38.07611356855978]
  Large language models (LLMs) have led to breakthroughs in language tasks, yet the internal mechanisms that enable their remarkable generalization and reasoning abilities remain opaque. This paper aims to uncover the mechanisms underlying LLM functionality through the lens of explainability.
  arXiv  Detail & Related papers  (2024-02-16T13:46:06Z)
• A Probabilistic Model Behind Self-Supervised Learning [53.64989127914936]
  In self-supervised learning (SSL), representations are learned via an auxiliary task without annotated labels. We present a generative latent variable model for self-supervised learning. We show that several families of discriminative SSL, including contrastive methods, induce a comparable distribution over representations.
  arXiv  Detail & Related papers  (2024-02-02T13:31:17Z)
• Continual Facial Expression Recognition: A Benchmark [3.181579197770883]
  This work presents the Continual Facial Expression Recognition (ConFER) benchmark that evaluates popular CL techniques on FER tasks. It presents a comparative analysis of several CL-based approaches on popular FER datasets such as CK+, RAF-DB, and AffectNet. CL techniques, under different learning settings, are shown to achieve state-of-the-art (SOTA) performance across several datasets.
  arXiv  Detail & Related papers  (2023-05-10T20:35:38Z)
• Reason from Context with Self-supervised Learning [15.16197896174348]
  We propose a new Self-supervised method with external memories for Context Reasoning (SeCo) In both tasks, SeCo outperformed all state-of-the-art (SOTA) SSL methods by a significant margin. Our results demonstrate that SeCo exhibits human-like behaviors.
  arXiv  Detail & Related papers  (2022-11-23T10:02:05Z)
• Variational Empowerment as Representation Learning for Goal-Based Reinforcement Learning [114.07623388322048]
  We discuss how the standard goal-conditioned RL (GCRL) is encapsulated by the objective variational empowerment. Our work lays a novel foundation from which to evaluate, analyze, and develop representation learning techniques in goal-based RL.
  arXiv  Detail & Related papers  (2021-06-02T18:12:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.