IDIL: Imitation Learning of Intent-Driven Expert Behavior

• URL: http://arxiv.org/abs/2404.16989v1
• Date: Thu, 25 Apr 2024 19:18:30 GMT
• Title: IDIL: Imitation Learning of Intent-Driven Expert Behavior
• Authors: Sangwon Seo, Vaibhav Unhelkar,
• Abstract summary: We introduce IDIL, a novel imitation learning algorithm to mimic diverse intent-driven behaviors of experts. It is capable of addressing sequential tasks with high-dimensional state representations. As it creates a generative model, IDIL demonstrates superior performance in intent inference metrics.
• Score: 2.07180164747172
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: When faced with accomplishing a task, human experts exhibit intentional behavior. Their unique intents shape their plans and decisions, resulting in experts demonstrating diverse behaviors to accomplish the same task. Due to the uncertainties encountered in the real world and their bounded rationality, experts sometimes adjust their intents, which in turn influences their behaviors during task execution. This paper introduces IDIL, a novel imitation learning algorithm to mimic these diverse intent-driven behaviors of experts. Iteratively, our approach estimates expert intent from heterogeneous demonstrations and then uses it to learn an intent-aware model of their behavior. Unlike contemporary approaches, IDIL is capable of addressing sequential tasks with high-dimensional state representations, while sidestepping the complexities and drawbacks associated with adversarial training (a mainstay of related techniques). Our empirical results suggest that the models generated by IDIL either match or surpass those produced by recent imitation learning benchmarks in metrics of task performance. Moreover, as it creates a generative model, IDIL demonstrates superior performance in intent inference metrics, crucial for human-agent interactions, and aptly captures a broad spectrum of expert behaviors.

Related papers

• Inferring Versatile Behavior from Demonstrations by Matching Geometric Descriptors [72.62423312645953]
  Humans intuitively solve tasks in versatile ways, varying their behavior in terms of trajectory-based planning and for individual steps. Current Imitation Learning algorithms often only consider unimodal expert demonstrations and act in a state-action-based setting. Instead, we combine a mixture of movement primitives with a distribution matching objective to learn versatile behaviors that match the expert's behavior and versatility.
  arXiv  Detail & Related papers  (2022-10-17T16:42:59Z)
• Causal Imitation Learning with Unobserved Confounders [82.22545916247269]
  We study imitation learning when sensory inputs of the learner and the expert differ. We show that imitation could still be feasible by exploiting quantitative knowledge of the expert trajectories.
  arXiv  Detail & Related papers  (2022-08-12T13:29:53Z)
• Chain of Thought Imitation with Procedure Cloning [129.62135987416164]
  We propose procedure cloning, which applies supervised sequence prediction to imitate the series of expert computations. We show that imitating the intermediate computations of an expert's behavior enables procedure cloning to learn policies exhibiting significant generalization to unseen environment configurations.
  arXiv  Detail & Related papers  (2022-05-22T13:14:09Z)
• Diverse Imitation Learning via Self-Organizing Generative Models [6.783186172518836]
  Imitation learning is the task of replicating expert policy from demonstrations, without access to a reward function. We adopt an encoder-free generative model for behavior cloning (BC) to accurately distinguish and imitate different modes. We show that our method significantly outperforms the state of the art across multiple experiments.
  arXiv  Detail & Related papers  (2022-05-06T21:55:31Z)
• Imitation by Predicting Observations [17.86983397979034]
  We present a new method for imitation solely from observations that achieves comparable performance to experts on challenging continuous control tasks. Our method, which we call FORM, is derived from an inverse RL objective and imitates using a model of expert behavior learned by generative modelling of the expert's observations. We show that FORM performs comparably to a strong baseline IRL method (GAIL) on the DeepMind Control Suite benchmark, while outperforming GAIL in the presence of task-irrelevant features.
  arXiv  Detail & Related papers  (2021-07-08T14:09:30Z)
• Seeing Differently, Acting Similarly: Imitation Learning with Heterogeneous Observations [126.78199124026398]
  In many real-world imitation learning tasks, the demonstrator and the learner have to act in different but full observation spaces. In this work, we model the above learning problem as Heterogeneous Observations Learning (HOIL) We propose the Importance Weighting with REjection (IWRE) algorithm based on the techniques of importance-weighting, learning with rejection, and active querying to solve the key challenge of occupancy measure matching.
  arXiv  Detail & Related papers  (2021-06-17T05:44:04Z)
• Stochastic Action Prediction for Imitation Learning [1.6385815610837169]
  Imitation learning is a data-driven approach to acquiring skills that relies on expert demonstrations to learn a policy that maps observations to actions. We demonstrate inherentity in demonstrations collected for tasks including line following with a remote-controlled car. We find that accounting for adversariality in the expert data leads to substantial improvement in the success rate of task completion.
  arXiv  Detail & Related papers  (2020-12-26T08:02:33Z)
• Learning "What-if" Explanations for Sequential Decision-Making [92.8311073739295]
  Building interpretable parameterizations of real-world decision-making on the basis of demonstrated behavior is essential. We propose learning explanations of expert decisions by modeling their reward function in terms of preferences with respect to "what if" outcomes. We highlight the effectiveness of our batch, counterfactual inverse reinforcement learning approach in recovering accurate and interpretable descriptions of behavior.
  arXiv  Detail & Related papers  (2020-07-02T14:24:17Z)
• Plausible Counterfactuals: Auditing Deep Learning Classifiers with Realistic Adversarial Examples [84.8370546614042]
  Black-box nature of Deep Learning models has posed unanswered questions about what they learn from data. Generative Adversarial Network (GAN) and multi-objectives are used to furnish a plausible attack to the audited model. Its utility is showcased within a human face classification task, unveiling the enormous potential of the proposed framework.
  arXiv  Detail & Related papers  (2020-03-25T11:08:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.