Focused Skill Discovery: Learning to Control Specific State Variables while Minimizing Side Effects

• URL: http://arxiv.org/abs/2510.04901v1
• Date: Mon, 06 Oct 2025 15:17:46 GMT
• Title: Focused Skill Discovery: Learning to Control Specific State Variables while Minimizing Side Effects
• Authors: Jonathan Colaço Carr, Qinyi Sun, Cameron Allen,
• Abstract summary: skill discovery algorithms often overlook the natural state variables present in reinforcement learning problems.<n>We introduce a general method that enables these skill discovery algorithms to learn skills that target and control specific state variables.<n>Our approach improves state space coverage by a factor of three, unlocks new learning capabilities, and automatically avoids negative side effects in downstream tasks.
• Score: 3.0035365527953526
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Skills are essential for unlocking higher levels of problem solving. A common approach to discovering these skills is to learn ones that reliably reach different states, thus empowering the agent to control its environment. However, existing skill discovery algorithms often overlook the natural state variables present in many reinforcement learning problems, meaning that the discovered skills lack control of specific state variables. This can significantly hamper exploration efficiency, make skills more challenging to learn with, and lead to negative side effects in downstream tasks when the goal is under-specified. We introduce a general method that enables these skill discovery algorithms to learn focused skills -- skills that target and control specific state variables. Our approach improves state space coverage by a factor of three, unlocks new learning capabilities, and automatically avoids negative side effects in downstream tasks.

Related papers

• SUSD: Structured Unsupervised Skill Discovery through State Factorization [12.57032768854794]
  Unsupervised Skill Discovery (USD) aims to autonomously learn a diverse set of skills without relying on extrinsic rewards.<n>We introduce SUSD, a novel framework that harnesses the compositional structure of environments by factorizing the state space into independent components.<n>SUSD allocates distinct skill variables to different factors, enabling more fine-grained control on the skill discovery process.
  arXiv  Detail & Related papers  (2026-02-02T04:21:33Z)
• Efficient Skill Discovery via Regret-Aware Optimization [37.27136009415794]
  We frame skill discovery as a min-max game of skill generation and policy learning.<n>We propose a regret-aware method on top of temporal representation learning.<n>Our method achieves a 15% zero shot improvement in high-dimensional environments.
  arXiv  Detail & Related papers  (2025-06-26T06:45:59Z)
• SkiLD: Unsupervised Skill Discovery Guided by Factor Interactions [48.003320766433966]
  This work introduces Skill Discovery from Local Dependencies (Skild) Skild develops a novel skill learning objective that explicitly encourages the mastering of skills that induce different interactions within an environment. We evaluate Skild in several domains with challenging, long-horizon sparse reward tasks including a realistic simulated household robot domain.
  arXiv  Detail & Related papers  (2024-10-24T04:01:59Z)
• Disentangled Unsupervised Skill Discovery for Efficient Hierarchical Reinforcement Learning [39.991887534269445]
  Disentangled Unsupervised Skill Discovery (DUSDi) is a method for learning disentangled skills that can be efficiently reused to solve downstream tasks. DUSDi decomposes skills into disentangled components, where each skill component only affects one factor of the state space. DUSDi successfully learns disentangled skills, and significantly outperforms previous skill discovery methods when it comes to applying the learned skills to solve downstream tasks.
  arXiv  Detail & Related papers  (2024-10-15T04:13:20Z)
• Behavior Contrastive Learning for Unsupervised Skill Discovery [75.6190748711826]
  We propose a novel unsupervised skill discovery method through contrastive learning among behaviors. Under mild assumptions, our objective maximizes the MI between different behaviors based on the same skill. Our method implicitly increases the state entropy to obtain better state coverage.
  arXiv  Detail & Related papers  (2023-05-08T06:02:11Z)
• Controllability-Aware Unsupervised Skill Discovery [94.19932297743439]
  We introduce a novel unsupervised skill discovery method, Controllability-aware Skill Discovery (CSD), which actively seeks complex, hard-to-control skills without supervision. The key component of CSD is a controllability-aware distance function, which assigns larger values to state transitions that are harder to achieve with the current skills. Our experimental results in six robotic manipulation and locomotion environments demonstrate that CSD can discover diverse complex skills with no supervision.
  arXiv  Detail & Related papers  (2023-02-10T08:03:09Z)
• Learning Options via Compression [62.55893046218824]
  We propose a new objective that combines the maximum likelihood objective with a penalty on the description length of the skills. Our objective learns skills that solve downstream tasks in fewer samples compared to skills learned from only maximizing likelihood.
  arXiv  Detail & Related papers  (2022-12-08T22:34:59Z)
• Choreographer: Learning and Adapting Skills in Imagination [60.09911483010824]
  We present Choreographer, a model-based agent that exploits its world model to learn and adapt skills in imagination. Our method decouples the exploration and skill learning processes, being able to discover skills in the latent state space of the model. Choreographer is able to learn skills both from offline data, and by collecting data simultaneously with an exploration policy.
  arXiv  Detail & Related papers  (2022-11-23T23:31:14Z)
• Hierarchical Skills for Efficient Exploration [70.62309286348057]
  In reinforcement learning, pre-trained low-level skills have the potential to greatly facilitate exploration. Prior knowledge of the downstream task is required to strike the right balance between generality (fine-grained control) and specificity (faster learning) in skill design. We propose a hierarchical skill learning framework that acquires skills of varying complexity in an unsupervised manner.
  arXiv  Detail & Related papers  (2021-10-20T22:29:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.