Hierarchical Reinforcement Learning for Cooperative Air-Ground Delivery in Urban System

• URL: http://arxiv.org/abs/2602.12913v1
• Date: Fri, 13 Feb 2026 13:19:59 GMT
• Title: Hierarchical Reinforcement Learning for Cooperative Air-Ground Delivery in Urban System
• Abstract summary: We propose HRL4AG, a Hierarchical Reinforcement Learning framework for cooperative Air-Ground delivery.<n>Specifically, HRL4AG employs a high-level manager to tackle the scalability bottleneck by decomposing the joint action space.<n>Experiments on two real-world datasets and an evaluation platform demonstrate that HRL4AG significantly outperforms state-of-the-art baselines.
• Score: 41.96919926267983
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Cooperative air-ground delivery has emerged as a promising logistics paradigm by leveraging the complementary strengths of UAVs and ground carriers. However, effective dispatching in such heterogeneous systems faces two critical challenges: i) the heterogeneity between flight and road dynamics, ii) the scalability bottleneck raised by the exponential decision variables in large-scale fleets. To address these challenges, we propose HRL4AG, a Hierarchical Reinforcement Learning framework for cooperative Air-Ground delivery. Specifically, HRL4AG employs a high-level manager to tackle the scalability bottleneck by decomposing the joint action space, and mode-specific workers that encode distinct flight and road dynamics to address the heterogeneity. Furthermore, a novel internal reward mechanism is designed to guide the hierarchical policy learning, addressing the credit assignment problem in sparse-reward settings. Extensive experiments on two real-world datasets and an evaluation platform demonstrate that HRL4AG significantly outperforms state-of-the-art baselines, improving the delivery success rate by up to 26% while achieving an 80-fold increase in computational efficiency.

Related papers

• WAM-Diff2: Hierarchical AR-to-Diffusion Distillation for Highly Efficient Autonomous Driving VLA [27.102605117827398]
  Vision-Language-Action (VLA) models have emerged as a prominent paradigm for end-to-end autonomous driving.<n>They are constrained by high computational latency and exposure bias arising from sequential autoregressive decoding.<n>We introduce WAM-Diff2, a multi-task discrete diffusion VLA framework powered by a three-stage hierarchical strategy.
  arXiv  Detail & Related papers  (2026-08-02T06:45:16Z)
• Reinforcement Learning for Delivery Drone-Based Participatory Sensing in Dynamic Environments [31.33963902763075]
  We formalize the problem as SensUAV and propose a Two TimeScale Reinforcement Learning framework (TSRL)<n>At the macro level, a task-em sensing dispatcher handles scalability by separately encoding distinct task features.<n>At the micro level, a wind-aware velocity controller learns fine-grained velocity scheduling to adapt to dynamic environmental variations.
  arXiv  Detail & Related papers  (2026-07-21T09:06:23Z)
• EnvRL: Learn from Environment Dynamics in Agentic Reinforcement Learning [78.62829041672663]
  Reinforcement learning (RL) has emerged as a powerful paradigm for training Large Language Models (LLMs) as agents.<n>We propose EnvRL, a framework that incorporates environment dynamics learning into agentic RL via two auxiliary objectives.
  arXiv  Detail & Related papers  (2026-06-16T08:48:09Z)
• Towards Precise Intent-Aligned VLA Aerial Navigation via Expert-Guided GRPO [10.368821738661707]
  We introduce an efficient reinforcement learning framework for VLA-based aerial navigation.<n>At its core, we propose EG-GRPO to augment online rollouts with few-shot expert data.<n>Across multiple tasks specified by complex human intents, EG-GRPO improves the success rate to 2.13x that of the SFT baseline, while improving intent alignment performance by 60.9%.
  arXiv  Detail & Related papers  (2026-06-01T14:31:35Z)
• RAD-2: Scaling Reinforcement Learning in a Generator-Discriminator Framework [49.531001563853984]
  RAD-2 is a unified generator-discriminator framework for closed-loop planning.<n>It produces diverse trajectory candidates, while an RL-optimized discriminator reranks these candidates according to their long-term driving quality.<n>It reduces the collision rate by 56% compared with strong diffusion-based planners.
  arXiv  Detail & Related papers  (2026-04-16T17:59:44Z)
• Multi-AUV Cooperative Target Tracking Based on Supervised Diffusion-Aided Multi-Agent Reinforcement Learning [19.584329065489893]
  This paper proposes a hierarchical MARL architecture comprising four layers: global training scheduling, multi-agent coordination, local decision-making, and real-time execution.<n>We propose the Supervised Diffusion-Aided MARL algorithm featuring three innovations: 1) a dual-decision architecture with segregated experience pools mitigating nonstationarity through structured experience replay; 2) a supervised learning mechanism guiding the diffusion model's reverse denoising process to generate high-fidelity training samples that accelerate convergence; and 3) disturbance-robust policy learning incorporating behavioral cloning loss to guide the Deep Deterministic Policy Gradient network update using high-quality replay actions, eliminating
  arXiv  Detail & Related papers  (2026-03-31T08:32:32Z)
• Hybrid Belief Reinforcement Learning for Efficient Coordinated Spatial Exploration [3.0222726254970174]
  Pure model-based approaches provide structured uncertainty estimates but lack adaptive policy learning.<n>This paper presents a hybrid belief-reinforcement learning framework to address this gap.<n>Results show 10.8% higher cumulative reward and 38% faster convergence over baselines.
  arXiv  Detail & Related papers  (2026-03-04T00:00:34Z)
• TopoCurate:Modeling Interaction Topology for Tool-Use Agent Training [53.93696896939915]
  Training tool-use agents typically rely on Supervised Fine-Tuning (SFT) on successful trajectories and Reinforcement Learning (RL) on pass-rate-selected tasks.<n>We propose TopoCurate, an interaction-aware framework that projects multi-trial rollouts from the same task into a unified semantic quotient topology.<n>TopoCurate achieves consistent gains of 4.2% (SFT) and 6.9% (RL) over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-03-02T10:38:54Z)
• Global Commander and Local Operative: A Dual-Agent Framework for Scene Navigation [96.88162755522342]
  Vision-and-Language Scene navigation is a fundamental capability for embodied human-AI.<n>We introduce DACo, a planning-grounding decoupled architecture that disentangles global deliberation from local grounding.<n>By disentangling global reasoning from local action, DACo alleviates cognitive overload and improves long-horizon stability.
  arXiv  Detail & Related papers  (2026-02-21T19:19:55Z)
• Multi-Agent DRL for V2X Resource Allocation: Disentangling Challenges and Benchmarking Solutions [51.22818149833102]
  Multi-agent reinforcement learning (MARL) has emerged as a promising approach for radio resource allocation in vehicle-to-everything (C-V2X) networks.<n>However, the multifaceted challenges inherent to MARL are often intertwined, making it difficult to understand their individual impact on performance in vehicular environments.<n>We bridge this gap by formulating C-V2X RRA as a sequence of multi-agent interference games with progressively increasing complexity.
  arXiv  Detail & Related papers  (2026-02-18T14:46:56Z)
• OmniVL-Guard: Towards Unified Vision-Language Forgery Detection and Grounding via Balanced RL [63.388513841293616]
  Existing forgery detection methods fail to handle the interleaved text, images, and videos prevalent in real-world misinformation.<n>To bridge this gap, this paper targets to develop a unified framework for omnibus vision-language forgery detection and grounding.<n>We propose textbf OmniVL-Guard, a balanced reinforcement learning framework for omnibus vision-language forgery detection and grounding.
  arXiv  Detail & Related papers  (2026-02-11T09:41:36Z)
• QoS-Aware Hierarchical Reinforcement Learning for Joint Link Selection and Trajectory Optimization in SAGIN-Supported UAV Mobility Management [52.15690855486153]
  A space-air-ground integrated network (SAGIN) has emerged as an essential architecture for enabling ubiquitous UAV connectivity.<n>This paper formulates UAV mobility management in SAGIN as a constrained multiobjective joint optimization problem.
  arXiv  Detail & Related papers  (2025-12-17T06:22:46Z)
• AirFed: Federated Graph-Enhanced Multi-Agent Reinforcement Learning for Multi-UAV Cooperative Mobile Edge Computing [21.4907371859268]
  Multiple Unmanned Aerial Vehicles (UAVs) cooperative Mobile Edge Computing (MEC) systems face critical challenges in coordinating trajectory planning, task offloading, and resource allocation.<n>Existing approaches suffer from limited scalability, slow convergence, and inefficient knowledge sharing among UAVs.<n>This paper proposes AirFed, a novel federated graph-enhanced multi-agent reinforcement learning framework.
  arXiv  Detail & Related papers  (2025-10-27T06:31:35Z)
• STAR-RIS-assisted Collaborative Beamforming for Low-altitude Wireless Networks [58.13757830013997]
  Wireless networks based on uncrewed aerial vehicles (UAVs) offer high mobility, flexibility, and coverage for urban communications.<n>They face severe signal attenuation in dense environments due to obstructions.<n>To address this critical issue, we consider introducing collaborative beam of UAVs and omni-directional re-altitude beamforming.
  arXiv  Detail & Related papers  (2025-10-25T01:28:37Z)
• Diffusion-RL Based Air Traffic Conflict Detection and Resolution Method [5.477141500588868]
  This paper proposes a novel autonomous conflict resolution framework named Diffusion-AC.<n>Our framework models its policy as a reverse denoising process guided by a value function, enabling it to generate a rich, high-quality, and multimodal action distribution.<n>Extensive simulation experiments demonstrate that the proposed method significantly outperforms a suite of state-of-the-art DRL benchmarks.
  arXiv  Detail & Related papers  (2025-09-02T23:17:46Z)
• CoordField: Coordination Field for Agentic UAV Task Allocation In Low-altitude Urban Scenarios [18.95579655633218]
  CoordField is a coordination field agent system for coordinating heterogeneous drone swarms in complex urban scenarios.<n>LLMs is responsible for interpreting high-level human instructions and converting them into executable commands for the UAV swarms.<n>A Coordination field mechanism is proposed to guide UAV motion and task selection, enabling decentralized and adaptive allocation of emergent tasks.
  arXiv  Detail & Related papers  (2025-04-30T18:02:45Z)
• Cluster-Based Multi-Agent Task Scheduling for Space-Air-Ground Integrated Networks [60.085771314013044]
  Low-altitude economy holds significant potential for development in areas such as communication and sensing.<n>We propose a Clustering-based Multi-agent Deep Deterministic Policy Gradient (CMADDPG) algorithm to address the multi-UAV cooperative task scheduling challenges in SAGIN.
  arXiv  Detail & Related papers  (2024-12-14T06:17:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.