Learn what matters: cross-domain imitation learning with task-relevant embeddings

• URL: http://arxiv.org/abs/2209.12093v1
• Date: Sat, 24 Sep 2022 21:56:58 GMT
• Title: Learn what matters: cross-domain imitation learning with task-relevant embeddings
• Authors: Tim Franzmeyer, Philip H. S. Torr, Jo\~ao F. Henriques
• Abstract summary: We study how an autonomous agent learns to perform a task from demonstrations in a different domain, such as a different environment or different agent. We propose a scalable framework that enables cross-domain imitation learning without access to additional demonstrations or further domain knowledge.
• Score: 77.34726150561087
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We study how an autonomous agent learns to perform a task from demonstrations in a different domain, such as a different environment or different agent. Such cross-domain imitation learning is required to, for example, train an artificial agent from demonstrations of a human expert. We propose a scalable framework that enables cross-domain imitation learning without access to additional demonstrations or further domain knowledge. We jointly train the learner agent's policy and learn a mapping between the learner and expert domains with adversarial training. We effect this by using a mutual information criterion to find an embedding of the expert's state space that contains task-relevant information and is invariant to domain specifics. This step significantly simplifies estimating the mapping between the learner and expert domains and hence facilitates end-to-end learning. We demonstrate successful transfer of policies between considerably different domains, without extra supervision such as additional demonstrations, and in situations where other methods fail.

Related papers

• RLIF: Interactive Imitation Learning as Reinforcement Learning [56.997263135104504]
  We show how off-policy reinforcement learning can enable improved performance under assumptions that are similar but potentially even more practical than those of interactive imitation learning. Our proposed method uses reinforcement learning with user intervention signals themselves as rewards. This relaxes the assumption that intervening experts in interactive imitation learning should be near-optimal and enables the algorithm to learn behaviors that improve over the potential suboptimal human expert.
  arXiv  Detail & Related papers  (2023-11-21T21:05:21Z)
• Learning with Style: Continual Semantic Segmentation Across Tasks and Domains [25.137859989323537]
  Domain adaptation and class incremental learning deal with domain and task variability separately, whereas their unified solution is still an open problem. We tackle both facets of the problem together, taking into account the semantic shift within both input and label spaces. We show how the proposed method outperforms existing approaches, which prove to be ill-equipped to deal with continual semantic segmentation under both task and domain shift.
  arXiv  Detail & Related papers  (2022-10-13T13:24:34Z)
• Label Distribution Learning for Generalizable Multi-source Person Re-identification [48.77206888171507]
  Person re-identification (Re-ID) is a critical technique in the video surveillance system. It is difficult to directly apply the supervised model to arbitrary unseen domains. We propose a novel label distribution learning (LDL) method to address the generalizable multi-source person Re-ID task.
  arXiv  Detail & Related papers  (2022-04-12T15:59:10Z)
• Improving the Generalization of Meta-learning on Unseen Domains via Adversarial Shift [3.1219977244201056]
  We propose a model-agnostic shift layer to learn how to simulate the domain shift and generate pseudo tasks. Based on the pseudo tasks, the meta-learning model can learn cross-domain meta-knowledge, which can generalize well on unseen domains.
  arXiv  Detail & Related papers  (2021-07-23T07:29:30Z)
• Disentangling Transfer and Interference in Multi-Domain Learning [53.34444188552444]
  We study the conditions where interference and knowledge transfer occur in multi-domain learning. We propose new metrics disentangling interference and transfer and set up experimental protocols. We demonstrate our findings on the CIFAR-100, MiniPlaces, and Tiny-ImageNet datasets.
  arXiv  Detail & Related papers  (2021-07-02T01:30:36Z)
• Cross-domain Imitation from Observations [50.669343548588294]
  Imitation learning seeks to circumvent the difficulty in designing proper reward functions for training agents by utilizing expert behavior. In this paper, we study the problem of how to imitate tasks when there exist discrepancies between the expert and agent MDP. We present a novel framework to learn correspondences across such domains.
  arXiv  Detail & Related papers  (2021-05-20T21:08:25Z)
• Domain-Robust Visual Imitation Learning with Mutual Information Constraints [0.0]
  We introduce a new algorithm called Disentangling Generative Adversarial Imitation Learning (DisentanGAIL) Our algorithm enables autonomous agents to learn directly from high dimensional observations of an expert performing a task.
  arXiv  Detail & Related papers  (2021-03-08T21:18:58Z)
• Domain Adaptive Knowledge Distillation for Driving Scene Semantic Segmentation [9.203485172547824]
  We present a novel approach to learn domain adaptive knowledge in models with limited memory. We propose a multi-level distillation strategy to effectively distil knowledge at different levels. We carry out extensive experiments and ablation studies on real-to-real as well as synthetic-to-real scenarios.
  arXiv  Detail & Related papers  (2020-11-03T03:01:09Z)
• Off-Dynamics Reinforcement Learning: Training for Transfer with Domain Classifiers [138.68213707587822]
  We propose a simple, practical, and intuitive approach for domain adaptation in reinforcement learning. We show that we can achieve this goal by compensating for the difference in dynamics by modifying the reward function. Our approach is applicable to domains with continuous states and actions and does not require learning an explicit model of the dynamics.
  arXiv  Detail & Related papers  (2020-06-24T17:47:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.