Beyond Conservative Automated Driving in Multi-Agent Scenarios via Coupled Model Predictive Control and Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2604.13891v1
• Date: Wed, 15 Apr 2026 13:58:38 GMT
• Title: Beyond Conservative Automated Driving in Multi-Agent Scenarios via Coupled Model Predictive Control and Deep Reinforcement Learning
• Abstract summary: Model Predictive Control (MPC) offers structured constraint handling through optimization.<n>Deep Reinforcement Learning (RL) learns adaptive behaviors from experience but often struggles with safety assurance.<n>We present an integrated MPC-RL framework to improve navigation performance in multi-agent scenarios.
• Score: 7.804905867413516
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automated driving at unsignalized intersections is challenging due to complex multi-vehicle interactions and the need to balance safety and efficiency. Model Predictive Control (MPC) offers structured constraint handling through optimization but relies on hand-crafted rules that often produce overly conservative behavior. Deep Reinforcement Learning (RL) learns adaptive behaviors from experience but often struggles with safety assurance and generalization to unseen environments. In this study, we present an integrated MPC-RL framework to improve navigation performance in multi-agent scenarios. Experiments show that MPC-RL outperforms standalone MPC and end-to-end RL across three traffic-density levels. Collectively, MPC-RL reduces the collision rate by 21% and improves the success rate by 6.5% compared to pure MPC. We further evaluate zero-shot transfer to a highway merging scenario without retraining. Both MPC-based methods transfer substantially better than end-to-end PPO, which highlights the role of the MPC backbone in cross-scenario robustness. The framework also shows faster loss stabilization than end-to-end RL during training, which indicates a reduced learning burden. These results suggest that the integrated approach can improve the balance between safety performance and efficiency in multi-agent intersection scenarios, while the MPC component provides a strong foundation for generalization across driving environments. The implementation code is available open-source.

Related papers

• SRL-MPC: Shape-Aware Reinforcement Learned Model Predictive Control [64.66285242901918]
  Shape-Aware Reinforcement Learned Model Predictive Control (SRL-MPC) is a method for safe, efficient, and adaptive navigation in crowds with heterogeneous shapes without geometry simplification.<n>A reinforcement learning framework then learns a neural policy that reads geometric separation features (GSFs) and outputs real-time MPC parameter updates.<n> Experiments in randomized crowd scenarios with arbitrary shaped robot fleets demonstrate the effectiveness, scalability, and robustness of SRL-MPC.
  arXiv  Detail & Related papers  (2026-08-21T14:46:06Z)
• Sharing the Control Authority Between Deep Reinforcement Learning and Model Predictive Control: Application to Multi-Class Transportation Networks [13.748063426728178]
  This paper proposes a novel DRL-MPC framework for multi-class transportation networks that divides control authority between DRL and MPC.<n>In the hierarchical framework, MPC operates at the higher level and determines low-frequency control inputs.<n>Results show that the proposed framework outperforms the hybrid state-feedback-MPC controller.
  arXiv  Detail & Related papers  (2026-08-21T08:26:20Z)
• Stable and Efficient Single-Rollout RL for Multimodal Reasoning [66.53652874617217]
  $textbfMSSR$ (Multimodal Stabilized Single-Rollout) is a group-free RLVR framework that achieves both stable optimization and effective multimodal reasoning performance.<n>In in-distribution evaluations, MSSR demonstrates superior training compute efficiency, achieving similar validation accuracy to the group-based baseline with half the training steps.
  arXiv  Detail & Related papers  (2025-12-20T05:07:53Z)
• COVLM-RL: Critical Object-Oriented Reasoning for Autonomous Driving Using VLM-Guided Reinforcement Learning [55.83415345423854]
  We propose COVLM-RL, a novel end-to-end driving framework that integrates Critical Object-oriented (CO) reasoning with reinforcement learning (RL)<n>Experiments conducted in the CARLA simulator demonstrate that COVLM-RL significantly improves the success rate by 30% in trained driving environments.
  arXiv  Detail & Related papers  (2025-12-10T06:18:16Z)
• Intersection of Reinforcement Learning and Bayesian Optimization for Intelligent Control of Industrial Processes: A Safe MPC-based DPG using Multi-Objective BO [0.0]
  Model Predictive Control (MPC)-based Reinforcement Learning (RL) offers a structured and interpretable alternative to Deep Neural Network (DNN)-based RL methods.<n>Standard MPC-RL approaches often suffer from slow convergence, suboptimal policy learning due to limited parameterization, and safety issues during online adaptation.<n>We propose a novel framework that integrates MPC-RL with Multi-Objective Bayesian Optimization (MOBO)
  arXiv  Detail & Related papers  (2025-07-14T02:31:52Z)
• TeLL-Drive: Enhancing Autonomous Driving with Teacher LLM-Guided Deep Reinforcement Learning [61.33599727106222]
  TeLL-Drive is a hybrid framework that integrates a Teacher LLM to guide an attention-based Student DRL policy.<n>A self-attention mechanism then fuses these strategies with the DRL agent's exploration, accelerating policy convergence and boosting robustness.
  arXiv  Detail & Related papers  (2025-02-03T14:22:03Z)
• Reinforcement Learning with Model Predictive Control for Highway Ramp Metering [14.389086937116582]
  This work explores the synergy between model-based and learning-based strategies to enhance traffic flow management.<n>The control problem is formulated as an RL task by crafting a suitable stage cost function.<n>An MPC-based RL approach, which leverages the MPC optimal problem as a function approximation for the RL algorithm, is proposed to learn to efficiently control an on-ramp.
  arXiv  Detail & Related papers  (2023-11-15T09:50:54Z)
• Safety Correction from Baseline: Towards the Risk-aware Policy in Robotics via Dual-agent Reinforcement Learning [64.11013095004786]
  We propose a dual-agent safe reinforcement learning strategy consisting of a baseline and a safe agent. Such a decoupled framework enables high flexibility, data efficiency and risk-awareness for RL-based control. The proposed method outperforms the state-of-the-art safe RL algorithms on difficult robot locomotion and manipulation tasks.
  arXiv  Detail & Related papers  (2022-12-14T03:11:25Z)
• Unified Automatic Control of Vehicular Systems with Reinforcement Learning [64.63619662693068]
  This article contributes a streamlined methodology for vehicular microsimulation. It discovers high performance control strategies with minimal manual design. The study reveals numerous emergent behaviors resembling wave mitigation, traffic signaling, and ramp metering.
  arXiv  Detail & Related papers  (2022-07-30T16:23:45Z)
• Model-predictive control and reinforcement learning in multi-energy system case studies [0.2810625954925815]
  We present an on-objective and off-policy multi- reinforcement learning (RL) approach against a linear model-predictive-control (LMPC) We show that a twin delayed deep deterministic policy gradient (TD3) RL agent offers potential to match and outperform the perfect foresight LMPC benchmark (101.5%) While in a more complex MES system configuration, the RL agent's performance is generally lower (94.6%), yet still better than the realistic LMPC (88.9%)
  arXiv  Detail & Related papers  (2021-04-20T06:51:50Z)
• Combining Reinforcement Learning with Model Predictive Control for On-Ramp Merging [10.480121529429631]
  Two broad classes of techniques have been proposed to solve motion planning problems in autonomous driving: Model Predictive Control (MPC) and Reinforcement Learning (RL) We first establish the strengths and weaknesses of state-of-the-art MPC and RL-based techniques through simulations. We subsequently present an algorithm which blends the model-free RL agent with the MPC solution and show that it provides better trade-offs between all metrics -- passenger comfort, efficiency, crash rate and robustness.
  arXiv  Detail & Related papers  (2020-11-17T07:42:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.