Offline Learning of Controllable Diverse Behaviors

• URL: http://arxiv.org/abs/2504.18160v1
• Date: Fri, 25 Apr 2025 08:16:56 GMT
• Title: Offline Learning of Controllable Diverse Behaviors
• Authors: Mathieu Petitbois, Rémy Portelas, Sylvain Lamprier, Ludovic Denoyer,
• Abstract summary: Imitation Learning (IL) techniques aim to replicate human behaviors in specific tasks.<n>We propose a new method based on temporal consistency and controllability.<n>We compare our approach to state-of-the-art methods over a diverse set of tasks and environments.
• Score: 19.0544729496907
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Imitation Learning (IL) techniques aim to replicate human behaviors in specific tasks. While IL has gained prominence due to its effectiveness and efficiency, traditional methods often focus on datasets collected from experts to produce a single efficient policy. Recently, extensions have been proposed to handle datasets of diverse behaviors by mainly focusing on learning transition-level diverse policies or on performing entropy maximization at the trajectory level. While these methods may lead to diverse behaviors, they may not be sufficient to reproduce the actual diversity of demonstrations or to allow controlled trajectory generation. To overcome these drawbacks, we propose a different method based on two key features: a) Temporal Consistency that ensures consistent behaviors across entire episodes and not just at the transition level as well as b) Controllability obtained by constructing a latent space of behaviors that allows users to selectively activate specific behaviors based on their requirements. We compare our approach to state-of-the-art methods over a diverse set of tasks and environments. Project page: https://mathieu-petitbois.github.io/projects/swr/

Related papers

• On Learning Informative Trajectory Embeddings for Imitation, Classification and Regression [19.01804572722833]
  In real-world sequential decision making tasks, learning from observed state-action trajectories is critical for tasks like imitation, classification, and clustering.<n>We propose a novel method for embedding state-action trajectories into a latent space that captures the skills and competencies in the dynamic underlying decision-making processes.
  arXiv  Detail & Related papers  (2025-01-16T06:52:58Z)
• SwitchCIT: Switching for Continual Instruction Tuning [14.085371250265224]
  Large language models (LLMs) and multimodal models (MMs) have exhibited impressive capabilities in various domains.<n>Continual instruction tuning is crucial to adapt a large model to evolving tasks and domains.<n>This work addresses the catastrophic forgetting in continual instruction learning through a mechanism for routing computations to parameter-efficient tuned models.
  arXiv  Detail & Related papers  (2024-07-16T14:37:33Z)
• Invariant Causal Imitation Learning for Generalizable Policies [87.51882102248395]
  We propose Invariant Causal Learning (ICIL) to learn an imitation policy. ICIL learns a representation of causal features that is disentangled from the specific representations of noise variables. We show that ICIL is effective in learning imitation policies capable of generalizing to unseen environments.
  arXiv  Detail & Related papers  (2023-11-02T16:52:36Z)
• Action-Quantized Offline Reinforcement Learning for Robotic Skill Learning [68.16998247593209]
  offline reinforcement learning (RL) paradigm provides recipe to convert static behavior datasets into policies that can perform better than the policy that collected the data. In this paper, we propose an adaptive scheme for action quantization. We show that several state-of-the-art offline RL methods such as IQL, CQL, and BRAC improve in performance on benchmarks when combined with our proposed discretization scheme.
  arXiv  Detail & Related papers  (2023-10-18T06:07:10Z)
• NormAUG: Normalization-guided Augmentation for Domain Generalization [60.159546669021346]
  We propose a simple yet effective method called NormAUG (Normalization-guided Augmentation) for deep learning. Our method introduces diverse information at the feature level and improves the generalization of the main path. In the test stage, we leverage an ensemble strategy to combine the predictions from the auxiliary path of our model, further boosting performance.
  arXiv  Detail & Related papers  (2023-07-25T13:35:45Z)
• Contrastive Example-Based Control [163.6482792040079]
  We propose a method for offline, example-based control that learns an implicit model of multi-step transitions, rather than a reward function. Across a range of state-based and image-based offline control tasks, our method outperforms baselines that use learned reward functions.
  arXiv  Detail & Related papers  (2023-07-24T19:43:22Z)
• Robust Task Representations for Offline Meta-Reinforcement Learning via Contrastive Learning [21.59254848913971]
  offline meta-reinforcement learning is a reinforcement learning paradigm that learns from offline data to adapt to new tasks. We propose a contrastive learning framework for task representations that are robust to the distribution of behavior policies in training and test. Experiments on a variety of offline meta-reinforcement learning benchmarks demonstrate the advantages of our method over prior methods.
  arXiv  Detail & Related papers  (2022-06-21T14:46:47Z)
• Learning Transferable Motor Skills with Hierarchical Latent Mixture Policies [37.09286945259353]
  We propose an approach to learn abstract motor skills from data using a hierarchical mixture latent variable model. We demonstrate in manipulation domains that the method can effectively cluster offline data into distinct, executable behaviours.
  arXiv  Detail & Related papers  (2021-12-09T17:37:14Z)
• Object-Aware Regularization for Addressing Causal Confusion in Imitation Learning [131.1852444489217]
  This paper presents Object-aware REgularizatiOn (OREO), a technique that regularizes an imitation policy in an object-aware manner. Our main idea is to encourage a policy to uniformly attend to all semantic objects, in order to prevent the policy from exploiting nuisance variables strongly correlated with expert actions.
  arXiv  Detail & Related papers  (2021-10-27T01:56:23Z)
• A Survey on Contrastive Self-supervised Learning [0.0]
  Self-supervised learning has gained popularity because of its ability to avoid the cost of annotating large-scale datasets. Contrastive learning has recently become a dominant component in self-supervised learning methods for computer vision, natural language processing (NLP), and other domains. This paper provides an extensive review of self-supervised methods that follow the contrastive approach.
  arXiv  Detail & Related papers  (2020-10-31T21:05:04Z)
• Task-Feature Collaborative Learning with Application to Personalized Attribute Prediction [166.87111665908333]
  We propose a novel multi-task learning method called Task-Feature Collaborative Learning (TFCL) Specifically, we first propose a base model with a heterogeneous block-diagonal structure regularizer to leverage the collaborative grouping of features and tasks. As a practical extension, we extend the base model by allowing overlapping features and differentiating the hard tasks.
  arXiv  Detail & Related papers  (2020-04-29T02:32:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.