Leveraging Human Feedback for Semantically-Relevant Skill Discovery

• URL: http://arxiv.org/abs/2604.24127v1
• Date: Mon, 27 Apr 2026 07:28:28 GMT
• Title: Leveraging Human Feedback for Semantically-Relevant Skill Discovery
• Abstract summary: We propose Semantically Relevant Skill Discovery (SRSD) as a novel human-in-the-loop approach to skill discovery.<n>SRSD collects semantic labels from human feedback and learns a reward function to encourage skills to be more semantically diverse and relevant.<n>We demonstrate that SRSD can improve semantic diversity and discover relevant behaviours while scaling effectively to a large variety of behaviours.
• Score: 11.543807097834785
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Unsupervised skill discovery in reinforcement learning aims to intrinsically motivate agents to discover diverse and useful behaviours. However, unconstrained approaches can produce unsafe, unethical, or misaligned behaviours. To mitigate these risks and improve the practical desireability of discovered skills, recent work grounds the discovery process by leveraging human preference feedback. However, preference-based approaches are feedback-inefficient and inherently ill-equipped to deal with skill spaces composed of a variety of different skills such as running, jumping, walking, etc. To overcome this limitation, we introduce semantic labelling, a novel and feedback-efficient approach that leverages human cognitive strengths to identify and label semantically meaningful behaviours. Based on semantic labelling, we propose Semantically Relevant Skill Discovery (SRSD), a novel human-in-the-loop approach that collects semantic labels from human feedback and learns a reward function to encourage skills to be more semantically diverse and relevant. Through our experiments in a 2D navigation environment and four locomotion environments, we demonstrate that SRSD can improve semantic diversity and discover relevant behaviours while scaling effectively to a large variety of behaviours.

Related papers

• Informal Learning Emerges in Everyday Human-LLM Interaction [74.53081471113929]
  We show that everyday human-LLM interaction contains measurable, selective and conditionally organized behavioural signatures of informal learning.<n>They shift AI evaluation from answer-delivery efficiency toward the preservation of cognitive opportunities for users to reason, test ideas and construct understanding.
  arXiv  Detail & Related papers  (2026-07-20T07:56:41Z)
• COLLIE: Guiding Skill Discovery in Semantically Coherent Latent Space [15.241002193763087]
  Unsupervised skill discovery (USD) aims to learn diverse behaviors without reward functions, but often results in task-irrelevant or hazardous behaviors.<n>We propose COLLIE, a framework that leverages dense unsupervised data to construct a semantically coherent skill latent space.<n>Our experiments show that COLLIE learns diverse, human-aligned skills, avoids hazardous behaviors, and achieves superior downstream performance with minimal human feedback.
  arXiv  Detail & Related papers  (2026-05-31T02:04:35Z)
• Act, Sense, Act: Learning Non-Markovian Active Perception Strategies from Large-Scale Egocentric Human Data [30.493131697188627]
  We formalize active perception as a non-Markovian process driven by information gain and decision branching.<n>We introduce CoMe-VLA, a cognitive and memory-aware vision-language-action framework.<n>Our framework integrates a cognitive auxiliary head for autonomous sub-task transitions and a dual-track memory system to maintain consistent self and environmental awareness.
  arXiv  Detail & Related papers  (2026-02-04T14:28:14Z)
• Human Behavior Atlas: Benchmarking Unified Psychological and Social Behavior Understanding [50.34821397821815]
  Human Behavior Atlas comprises over 100,000 samples spanning text, audio, and visual modalities.<n>Our unification efforts can reduce redundancy and cost, enable training to scale efficiently across tasks, and enhance generalization of behavioral features across domains.
  arXiv  Detail & Related papers  (2025-10-06T15:16:45Z)
• Truly Self-Improving Agents Require Intrinsic Metacognitive Learning [59.60803539959191]
  Self-improving agents aim to continuously acquire new capabilities with minimal supervision.<n>Current approaches face two key limitations: their self-improvement processes are often rigid, fail to generalize across tasks domains, and struggle to scale with increasing agent capabilities.<n>We argue that effective self-improvement requires intrinsic metacognitive learning, defined as an agent's intrinsic ability to actively evaluate, reflect on, and adapt its own learning processes.
  arXiv  Detail & Related papers  (2025-06-05T14:53:35Z)
• Human-Aligned Skill Discovery: Balancing Behaviour Exploration and Alignment [14.948610521764415]
  We propose Human-aligned Skill Discovery (HaSD) to discover safer, more aligned skills.<n>HaSD simultaneously optimises skill diversity and alignment with human values.<n>We demonstrate its effectiveness in both 2D navigation and SafetyGymnasium environments.
  arXiv  Detail & Related papers  (2025-01-29T06:14:27Z)
• Visual-Geometric Collaborative Guidance for Affordance Learning [63.038406948791454]
  We propose a visual-geometric collaborative guided affordance learning network that incorporates visual and geometric cues. Our method outperforms the representative models regarding objective metrics and visual quality.
  arXiv  Detail & Related papers  (2024-10-15T07:35:51Z)
• Language Guided Skill Discovery [56.84356022198222]
  We introduce Language Guided Skill Discovery (LGSD) to maximize semantic diversity between skills.<n>LGSD takes user prompts as input and outputs a set of semantically distinctive skills.<n>We demonstrate that LGSD enables legged robots to visit different user-intended areas on a plane by simply changing the prompt.
  arXiv  Detail & Related papers  (2024-06-07T04:25:38Z)
• ComSD: Balancing Behavioral Quality and Diversity in Unsupervised Skill Discovery [12.277005054008017]
  We propose textbfContrastive dynatextbfmic textbfSkill textbfDiscovery textbf(ComSD).<n>ComSD generates diverse and exploratory unsupervised skills through a novel intrinsic incentive, named contrastive dynamic reward.<n>It can also discover distinguishable and far-reaching exploration skills in the challenging tree-like 2D maze.
  arXiv  Detail & Related papers  (2023-09-29T12:53:41Z)
• Controlled Diversity with Preference : Towards Learning a Diverse Set of Desired Skills [15.187171070594935]
  We propose Controlled Diversity with Preference (CDP), a collaborative human-guided mechanism for an agent to learn a set of skills that is diverse as well as desirable. The key principle is to restrict the discovery of skills to those regions that are deemed to be desirable as per a preference model trained using human preference labels on trajectory pairs. We evaluate our approach on 2D navigation and Mujoco environments and demonstrate the ability to discover diverse, yet desirable skills.
  arXiv  Detail & Related papers  (2023-03-07T03:37:47Z)
• Flexible social inference facilitates targeted social learning when rewards are not observable [58.762004496858836]
  Groups coordinate more effectively when individuals are able to learn from others' successes. We suggest that social inference capacities may help bridge this gap, allowing individuals to update their beliefs about others' underlying knowledge and success from observable trajectories of behavior.
  arXiv  Detail & Related papers  (2022-12-01T21:04:03Z)
• Relative Variational Intrinsic Control [11.328970848714919]
  Relative Variational Intrinsic Control (RVIC) incentivizes learning skills that are distinguishable in how they change the agent's relationship to its environment. We show how RVIC skills are more useful than skills discovered by existing methods when used in hierarchical reinforcement learning.
  arXiv  Detail & Related papers  (2020-12-14T18:59:23Z)
• Can Semantic Labels Assist Self-Supervised Visual Representation Learning? [194.1681088693248]
  We present a new algorithm named Supervised Contrastive Adjustment in Neighborhood (SCAN) In a series of downstream tasks, SCAN achieves superior performance compared to previous fully-supervised and self-supervised methods. Our study reveals that semantic labels are useful in assisting self-supervised methods, opening a new direction for the community.
  arXiv  Detail & Related papers  (2020-11-17T13:25:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.