Bridging Large-Model Reasoning and Real-Time Control via Agentic Fast-Slow Planning

• URL: http://arxiv.org/abs/2604.01681v1
• Date: Thu, 02 Apr 2026 06:34:29 GMT
• Title: Bridging Large-Model Reasoning and Real-Time Control via Agentic Fast-Slow Planning
• Abstract summary: Agentic Fast-Slow Planning is a hierarchical framework that decouples perception, reasoning, planning, and control across natural timescales.<n>We show that Agentic Fast-Slow Planning improves robustness under perturbations, reducing lateral deviation by up to 45% and completion time by over 12% compared to pure MPC and an A*-guided MPC baseline.
• Score: 42.15812524999639
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large foundation models enable powerful reasoning for autonomous systems, but mapping semantic intent to reliable real-time control remains challenging. Existing approaches either (i) let Large Language Models (LLMs) generate trajectories directly - brittle, hard to verify, and latency-prone - or (ii) adjust Model Predictive Control (MPC) objectives online - mixing slow deliberation with fast control and blurring interfaces. We propose Agentic Fast-Slow Planning, a hierarchical framework that decouples perception, reasoning, planning, and control across natural timescales. The framework contains two bridges. Perception2Decision compresses scenes into ego-centric topologies using an on-vehicle Vision-Language Model (VLM) detector, then maps them to symbolic driving directives in the cloud with an LLM decision maker - reducing bandwidth and delay while preserving interpretability. Decision2Trajectory converts directives into executable paths: Semantic-Guided A* embeds language-derived soft costs into classical search to bias solutions toward feasible trajectories, while an Agentic Refinement Module adapts planner hyperparameters using feedback and memory. Finally, MPC tracks the trajectories in real time, with optional cloud-guided references for difficult cases. Experiments in CARLA show that Agentic Fast-Slow Planning improves robustness under perturbations, reducing lateral deviation by up to 45% and completion time by over 12% compared to pure MPC and an A*-guided MPC baseline. Code is available at https://github.com/cjychenjiayi/icra2026_AFSP.

Related papers

• MA-HEAD-Net: Adaptive Rule-Guided Multi-Agent DRL for AoI Minimization in UAV-Assisted Emergency Networks [71.01350912967922]
  In post-disaster scenarios, unmanned aerial vehicles (UAVs) are critical for establishing emergency communication networks.<n>This paper investigates age of information (AoI) minimization for UAV-assisted emergency communications with heterogeneous emergency services.<n>We develop MA-HEAD-Net, an adaptive rule-guided multi-agent deep reinforcement learning framework.
  arXiv  Detail & Related papers  (2026-08-02T10:05:50Z)
• Intelligent Multi-UAV Navigation in ITNTNs: A Hierarchical LLM Approach [33.68706070829591]
  Large Language Models (LLMs) excel at semantic reasoning but suffer from high inference latency, rendering them unsuitable for real-time aerodynamic control.<n>A massive cloud-based LLM deployed on a High-Altitude Platform Station manages slow-timescale global load balancing.<n> lightweight edge-LLMs on individual UAVs translate local observations into tactical sub-goals to execute collision-free, handover-aware trajectories.
  arXiv  Detail & Related papers  (2026-07-21T00:34:42Z)
• PEACE: A Planner-Executor Agent with Constraint Enforcement for UAVs [4.022353603224729]
  We propose a planner-executor agent for PX4-based drones that decouples high-level mission planning from low-level control.<n>A large language model performs single-pass task planning, while execution is handled through a structured ROS 2 tool-calling interface.<n>Results highlight improved explainability, constraint enforcement, and reduced LLM calls compared to tightly coupled LLM control.
  arXiv  Detail & Related papers  (2026-05-26T10:03:22Z)
• Open-Ended Instruction Realization with LLM-Enabled Multi-Planner Scheduling in Autonomous Vehicles [40.195332742819865]
  This study proposes an instruction-realization framework that leverages a large language model (LLM) to interpret instructions.<n>It generates executable scripts that schedule multiple model predictive control (MPC)-based motion planners based on real-time feedback, and converts planned trajectories into control signals.
  arXiv  Detail & Related papers  (2026-04-09T09:32:21Z)
• Implicit Maximum Likelihood Estimation for Real-time Generative Model Predictive Control [5.692340907039085]
  Implicit Maximum Likelihood Estimation (IMLE) is an alternative generative modeling approach for planning.<n>Our results demonstrate that IMLE achieves competitive performance on standard offline reinforcement learning benchmarks.<n>We further validate IMLE in a closed-loop human navigation scenario, operating in real-time, demonstrating how it enables rapid and adaptive plan generation.
  arXiv  Detail & Related papers  (2026-03-14T03:39:31Z)
• Direct Contact-Tolerant Motion Planning With Vision Language Models [64.48049143625063]
  Navigation in cluttered environments often requires robots to tolerate contact with movable or deformable objects.<n>We propose a direct contact-tolerant (DCT) planner, which integrates vision-language models (VLMs) into direct point perception and navigation.<n>We implement DCT in Isaac Sim and a real car-like robot, demonstrating that DCT achieves robust and efficient navigation in cluttered environments with movable obstacles.
  arXiv  Detail & Related papers  (2026-03-05T10:05:00Z)
• Tiny-Critic RAG: Empowering Agentic Fallback with Parameter-Efficient Small Language Models [7.704706624419061]
  Retrieval-Augmented Generation grounds Large Language Models (LLMs) to mitigate factual hallucinations.<n>We propose Tiny-Critic RAG, a parameter-efficient Small Language Model (SLM) via Low-Rank Adaptation (LoRA)<n>Tiny-Critic RAG achieves routing accuracy comparable to GPT-4o-mini while reducing latency by an order of magnitude.
  arXiv  Detail & Related papers  (2026-03-01T00:16:31Z)
• DLLM Agent: See Farther, Run Faster [94.74432470237817]
  Diffusion large language models (DLLMs) have emerged as an alternative to autoregressive (AR) decoding with appealing efficiency and modeling properties.<n>We study this in a controlled setting by instantiatingDLLM and AR backbones within the same agent workflow.<n>We find thatDLLM Agents are on average over 30% faster end to end than AR agents, with some cases exceeding 8x speedup.
  arXiv  Detail & Related papers  (2026-02-07T09:01:18Z)
• Deep Bilinear Koopman Model for Real-Time Vehicle Control in Frenet Frame [0.0]
  This paper presents a deep Koopman approach for modeling and control of vehicle dynamics within the curvilinear Frenet frame.<n>The proposed framework uses a deep neural network architecture to simultaneously learn the Koopman operator and its associated invariant subspace from the data.<n>The proposed controller achieved significant reductions in tracking error relative to baseline controllers, confirming its suitability for real-time implementation in embedded autonomous vehicle systems.
  arXiv  Detail & Related papers  (2025-07-16T18:49:44Z)
• Pangu Embedded: An Efficient Dual-system LLM Reasoner with Metacognition [95.54406667705999]
  Pangu Embedded is an efficient Large Language Model (LLM) reasoner developed on Ascend Neural Processing Units (NPUs)<n>It addresses the significant computational costs and inference latency challenges prevalent in existing reasoning-optimized LLMs.<n>It delivers rapid responses and state-of-the-art reasoning quality within a single, unified model architecture.
  arXiv  Detail & Related papers  (2025-05-28T14:03:02Z)
• SCoTT: Strategic Chain-of-Thought Tasking for Wireless-Aware Robot Navigation in Digital Twins [78.53885607559958]
  We propose SCoTT, a wireless-aware path planning framework.<n>We show that SCoTT achieves path gains within 2% of DP-WA* while consistently generating shorter trajectories.<n>We also show the practical viability of our approach by deploying SCoTT as a ROS node within Gazebo simulations.
  arXiv  Detail & Related papers  (2024-11-27T10:45:49Z)
• DiFSD: Ego-Centric Fully Sparse Paradigm with Uncertainty Denoising and Iterative Refinement for Efficient End-to-End Self-Driving [55.53171248839489]
  We propose an ego-centric fully sparse paradigm, named DiFSD, for end-to-end self-driving.<n>Specifically, DiFSD mainly consists of sparse perception, hierarchical interaction and iterative motion planner.<n>Experiments conducted on nuScenes and Bench2Drive datasets demonstrate the superior planning performance and great efficiency of DiFSD.
  arXiv  Detail & Related papers  (2024-09-15T15:55:24Z)
• SIMPL: A Simple and Efficient Multi-agent Motion Prediction Baseline for Autonomous Driving [27.776472262857045]
  This paper presents a Simple and effIcient Motion Prediction baseLine (SIMPL) for autonomous vehicles. We propose a compact and efficient global feature fusion module that performs directed message passing in a symmetric manner. As a strong baseline, SIMPL exhibits highly competitive performance on Argoverse 1 & 2 motion forecasting benchmarks.
  arXiv  Detail & Related papers  (2024-02-04T15:07:49Z)
• Trajectory Planning for Autonomous Vehicles Using Hierarchical Reinforcement Learning [21.500697097095408]
  Planning safe trajectories under uncertain and dynamic conditions makes the autonomous driving problem significantly complex. Current sampling-based methods such as Rapidly Exploring Random Trees (RRTs) are not ideal for this problem because of the high computational cost. We propose a Hierarchical Reinforcement Learning structure combined with a Proportional-Integral-Derivative (PID) controller for trajectory planning.
  arXiv  Detail & Related papers  (2020-11-09T20:49:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.