CoMI-IRL: Contrastive Multi-Intention Inverse Reinforcement Learning

• URL: http://arxiv.org/abs/2602.07496v1
• Date: Sat, 07 Feb 2026 11:14:34 GMT
• Title: CoMI-IRL: Contrastive Multi-Intention Inverse Reinforcement Learning
• Authors: Antonio Mone, Frans A. Oliehoek, Luciano Cavalcante Siebert,
• Abstract summary: Inverse Reinforcement Learning seeks to infer reward functions from expert demonstrations.<n>Recent deep generative MI-IRL approaches couple behavior clustering and reward learning, but typically require prior knowledge of the number of true behavioral modes.<n>We propose Contrastive Multi-Intention IRL (CoMI-IRL), a transformer-based unsupervised framework that decouples behavior representation and clustering from downstream reward learning.
• Score: 9.508688798968809
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Inverse Reinforcement Learning (IRL) seeks to infer reward functions from expert demonstrations. When demonstrations originate from multiple experts with different intentions, the problem is known as Multi-Intention IRL (MI-IRL). Recent deep generative MI-IRL approaches couple behavior clustering and reward learning, but typically require prior knowledge of the number of true behavioral modes $K^*$. This reliance on expert knowledge limits their adaptability to new behaviors, and only enables analysis related to the learned rewards, and not across the behavior modes used to train them. We propose Contrastive Multi-Intention IRL (CoMI-IRL), a transformer-based unsupervised framework that decouples behavior representation and clustering from downstream reward learning. Our experiments show that CoMI-IRL outperforms existing approaches without a priori knowledge of $K^*$ or labels, while allowing for visual interpretation of behavior relationships and adaptation to unseen behavior without full retraining.

Related papers

• Non-Adversarial Inverse Reinforcement Learning via Successor Feature Matching [23.600285251963395]
  In inverse reinforcement learning (IRL), an agent seeks to replicate expert demonstrations through interactions with the environment.<n>Traditionally, IRL is treated as an adversarial game, where an adversary searches over reward models, and a learner optimize the reward through repeated RL procedures.<n>We propose a novel approach to IRL by direct policy optimization, exploiting a linear factorization of the return as the inner product of successor features and a reward vector.
  arXiv  Detail & Related papers  (2024-11-11T14:05:50Z)
• RILe: Reinforced Imitation Learning [60.63173816209543]
  RILe (Reinforced Learning) is a framework that combines the strengths of imitation learning and inverse reinforcement learning to learn a dense reward function efficiently.<n>Our framework produces high-performing policies in high-dimensional tasks where direct imitation fails to replicate complex behaviors.
  arXiv  Detail & Related papers  (2024-06-12T17:56:31Z)
• Multi-intention Inverse Q-learning for Interpretable Behavior Representation [12.135423420992334]
  inverse reinforcement learning (IRL) methods have proven instrumental in reconstructing animal's intentions underlying complex behaviors. We introduce the class of hierarchical inverse Q-learning (HIQL) algorithms. Applying HIQL to simulated experiments and several real animal behavior datasets, our approach outperforms current benchmarks in behavior prediction.
  arXiv  Detail & Related papers  (2023-11-23T09:27:08Z)
• BC-IRL: Learning Generalizable Reward Functions from Demonstrations [51.535870379280155]
  inverse reinforcement learning method learns reward functions that generalize better when compared to maximum-entropy IRL approaches. We show that BC-IRL learns rewards that generalize better on an illustrative simple task and two continuous robotic control tasks, achieving over twice the success rate of baselines in challenging generalization settings.
  arXiv  Detail & Related papers  (2023-03-28T17:57:20Z)
• Multiagent Inverse Reinforcement Learning via Theory of Mind Reasoning [0.0]
  We propose a novel approach to Multiagent Inverse Reinforcement Learning (MIRL) MIRL aims to infer the reward functions guiding the behavior of each individual given trajectories of a team's behavior during task performance. We evaluate our approach in a simulated 2-player search-and-rescue operation.
  arXiv  Detail & Related papers  (2023-02-20T19:07:42Z)
• CLARE: Conservative Model-Based Reward Learning for Offline Inverse Reinforcement Learning [26.05184273238923]
  This work aims to tackle a major challenge in offline Inverse Reinforcement Learning (IRL) We devise a principled algorithm (namely CLARE) that solves offline IRL efficiently via integrating "conservatism" into a learned reward function. Our theoretical analysis provides an upper bound on the return gap between the learned policy and the expert policy.
  arXiv  Detail & Related papers  (2023-02-09T17:16:29Z)
• Basis for Intentions: Efficient Inverse Reinforcement Learning using Past Experience [89.30876995059168]
  inverse reinforcement learning (IRL) -- inferring the reward function of an agent from observing its behavior. This paper addresses the problem of IRL -- inferring the reward function of an agent from observing its behavior.
  arXiv  Detail & Related papers  (2022-08-09T17:29:49Z)
• Active Exploration for Inverse Reinforcement Learning [58.295273181096036]
  We propose a novel IRL algorithm: Active exploration for Inverse Reinforcement Learning (AceIRL) AceIRL actively explores an unknown environment and expert policy to quickly learn the expert's reward function and identify a good policy. We empirically evaluate AceIRL in simulations and find that it significantly outperforms more naive exploration strategies.
  arXiv  Detail & Related papers  (2022-07-18T14:45:55Z)
• Transferable Reward Learning by Dynamics-Agnostic Discriminator Ensemble [8.857776147129464]
  Recovering reward function from expert demonstrations is a fundamental problem in reinforcement learning. We present a dynamics-agnostic discriminator-ensemble reward learning method capable of learning both state-action and state-only reward functions.
  arXiv  Detail & Related papers  (2022-06-01T05:16:39Z)
• Context-Hierarchy Inverse Reinforcement Learning [30.71220625227959]
  inverse reinforcement learning (IRL) agent learns to act intelligently by observing expert demonstrations and learning the expert's underlying reward function. We present Context Hierarchy IRL(CHIRL), a new IRL algorithm that exploits the context to scale up IRL and learn reward functions of complex behaviors. Experiments on benchmark tasks, including a large scale autonomous driving task in the CARLA simulator, show promising results in scaling up IRL for tasks with complex reward functions.
  arXiv  Detail & Related papers  (2022-02-25T10:29:05Z)
• PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse Temporal Difference Learning [102.36450942613091]
  We propose an inverse reinforcement learning algorithm, called emphinverse temporal difference learning (ITD) We show how to seamlessly integrate ITD with learning from online environment interactions, arriving at a novel algorithm for reinforcement learning with demonstrations, called $Psi Phi$-learning.
  arXiv  Detail & Related papers  (2021-02-24T21:12:09Z)
• Semi-supervised reward learning for offline reinforcement learning [71.6909757718301]
  Training agents usually requires reward functions, but rewards are seldom available in practice and their engineering is challenging and laborious. We propose semi-supervised learning algorithms that learn from limited annotations and incorporate unlabelled data. In our experiments with a simulated robotic arm, we greatly improve upon behavioural cloning and closely approach the performance achieved with ground truth rewards.
  arXiv  Detail & Related papers  (2020-12-12T20:06:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.