Split the Differences, Pool the Rest: Provably Efficient Multi-Objective Imitation

• URL: http://arxiv.org/abs/2605.12000v2
• Date: Mon, 18 May 2026 14:32:07 GMT
• Title: Split the Differences, Pool the Rest: Provably Efficient Multi-Objective Imitation
• Abstract summary: We introduce Multi-Output Augmented Behavioral Cloning (MA-BC)<n>MA-BC partitions divergent expert data while pooling state-action pairs where no behavior conflict is observed.<n>We establish a novel lower bound for multi-objective imitation learning, demonstrating that MA-BC is minimax optimal.
• Score: 49.86232017439639
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This work investigates multi-objective imitation learning: the problem of recovering policies that lie on the Pareto front given demonstrations from multiple Pareto-optimal experts in a Multi-Objective Markov Decision Process (MOMDP). Standard imitation approaches are ill-equipped for this regime, as naively aggregating conflicting expert trajectories can result in dominated policies. To address this, we introduce Multi-Output Augmented Behavioral Cloning (MA-BC), an algorithm that systematically partitions divergent expert data while pooling state-action pairs where no behavior conflict is observed. Theoretically, we prove that MA-BC converges to Pareto-optimal policies at a faster statistical rate than any learner that considers each expert dataset independently. Furthermore, we establish a novel lower bound for multi-objective imitation learning, demonstrating that MA-BC is minimax optimal. Finally, we empirically validate our algorithm across diverse discrete environments and, guided by our theoretical insights, extend and evaluate MA-BC on a continuous Linear Quadratic Regulator (LQR) control task.

Related papers

• Bayesian Anytime Pareto Set Identification for Multi-Objective Multi-Armed Bandits [0.37515646463759694]
  We introduce the first anytime Multi-Objective Multi-Armed Bandit algorithm for the Pareto Set Identification problem.<n>We demonstrate its practical utility in a challenging multi-objective molecular discovery setting.<n>We complement these empirical findings with a theoretical proof of the algorithm's correctness.
  arXiv  Detail & Related papers  (2026-06-17T07:56:51Z)
• Constrained Multi-Objective Reinforcement Learning with Max-Min Criterion [24.714655444299435]
  Multi-Objective Reinforcement Learning (MORL) extends standard RL by optimizing policies with respect to multiple, often conflicting, objectives.<n>We propose a MORL framework that integrates the max-min criterion with explicit constraint satisfaction.
  arXiv  Detail & Related papers  (2026-05-29T14:52:12Z)
• Multi-Objective Constraint Inference using Inverse reinforcement learning [4.8029888032686445]
  Multi-Objective Constraint Inference (MOCI) is a novel framework designed to jointly extract shared constraints and individual preferences from heterogeneous expert trajectories.<n>MOCI effectively models and learns from diverse, and potentially conflicting, behaviors.
  arXiv  Detail & Related papers  (2026-05-07T21:09:09Z)
• From Sparse Decisions to Dense Reasoning: A Multi-attribute Trajectory Paradigm for Multimodal Moderation [59.27094165576015]
  We propose a novel learning paradigm (UniMod) that transitions from sparse decision-making to dense reasoning traces.<n>By constructing structured trajectories encompassing evidence grounding, modality assessment, risk mapping, policy decision, and response generation, we reformulate monolithic decision tasks into a multi-dimensional boundary learning process.<n>We introduce specialized optimization strategies to decouple task-specific parameters and rebalance training dynamics, effectively resolving interference between diverse objectives in multi-task learning.
  arXiv  Detail & Related papers  (2026-01-28T09:29:40Z)
• Principled Multimodal Representation Learning [99.53621521696051]
  Multimodal representation learning seeks to create a unified representation space by integrating diverse data modalities.<n>Recent advances have investigated the simultaneous alignment of multiple modalities, yet several challenges remain.<n>We propose Principled Multimodal Representation Learning (PMRL), a novel framework that achieves simultaneous alignment of multiple modalities.
  arXiv  Detail & Related papers  (2025-07-23T09:12:25Z)
• Multimodal Mathematical Reasoning with Diverse Solving Perspective [65.07953438724105]
  We introduce MathV-DP, a novel dataset that captures multiple diverse solution trajectories for each image-question pair.<n>We propose Qwen-VL-DP, a model built upon Qwen-VL, fine-tuned with supervised learning and enhanced via group relative policy optimization.<n>Our method emphasizes learning from varied reasoning perspectives and distinguishing between correct yet distinct solutions.
  arXiv  Detail & Related papers  (2025-07-03T17:07:20Z)
• Preference-Based Multi-Agent Reinforcement Learning: Data Coverage and Algorithmic Techniques [65.55451717632317]
  We study Preference-Based Multi-Agent Reinforcement Learning (PbMARL)<n>We identify the Nash equilibrium from a preference-only offline dataset in general-sum games.<n>Our findings underscore the multifaceted approach required for PbMARL.
  arXiv  Detail & Related papers  (2024-09-01T13:14:41Z)
• Pareto Inverse Reinforcement Learning for Diverse Expert Policy Generation [6.876580618014666]
  In this paper, we adapt inverse reinforcement learning (IRL) by using reward distance estimates for regularizing the discriminator. We show that ParIRL outperforms other IRL algorithms for various multi-objective control tasks.
  arXiv  Detail & Related papers  (2024-08-22T03:51:39Z)
• Finite-Time Convergence and Sample Complexity of Actor-Critic Multi-Objective Reinforcement Learning [20.491176017183044]
  This paper tackles the multi-objective reinforcement learning (MORL) problem. It introduces an innovative actor-critic algorithm named MOAC which finds a policy by iteratively making trade-offs among conflicting reward signals.
  arXiv  Detail & Related papers  (2024-05-05T23:52:57Z)
• Multi-Objective Policy Gradients with Topological Constraints [108.10241442630289]
  We present a new algorithm for a policy gradient in TMDPs by a simple extension of the proximal policy optimization (PPO) algorithm. We demonstrate this on a real-world multiple-objective navigation problem with an arbitrary ordering of objectives both in simulation and on a real robot.
  arXiv  Detail & Related papers  (2022-09-15T07:22:58Z)
• Off-Policy Correction For Multi-Agent Reinforcement Learning [9.599347559588216]
  Multi-agent reinforcement learning (MARL) provides a framework for problems involving multiple interacting agents. Despite apparent similarity to the single-agent case, multi-agent problems are often harder to train and analyze theoretically. We propose MA-Trace, a new on-policy actor-critic algorithm, which extends V-Trace to the MARL setting.
  arXiv  Detail & Related papers  (2021-11-22T14:23:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.