Perception Helps Planning: Facilitating Multi-Stage Lane-Level Integration via Double-Edge Structures

• URL: http://arxiv.org/abs/2407.11644v2
• Date: Mon, 09 Dec 2024 13:09:26 GMT
• Title: Perception Helps Planning: Facilitating Multi-Stage Lane-Level Integration via Double-Edge Structures
• Abstract summary: Perception Helps Planning (PHP) reconciles lane-level planning with perception.<n> Experiments on three Carla benchmarks show significant improvements in driving score of 27.20%, 33.47%, and 15.54% over existing algorithms.
• Score: 13.648498083189931
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: When planning for autonomous driving, it is crucial to consider essential traffic elements such as lanes, intersections, traffic regulations, and dynamic agents. However, they are often overlooked by the traditional end-to-end planning methods, likely leading to inefficiencies and non-compliance with traffic regulations. In this work, we endeavor to integrate the perception of these elements into the planning task. To this end, we propose Perception Helps Planning (PHP), a novel framework that reconciles lane-level planning with perception. This integration ensures that planning is inherently aligned with traffic constraints, thus facilitating safe and efficient driving. Specifically, PHP focuses on both edges of a lane for planning and perception purposes, taking into consideration the 3D positions of both lane edges and attributes for lane intersections, lane directions, lane occupancy, and planning. In the algorithmic design, the process begins with the transformer encoding multi-camera images to extract the above features and predicting lane-level perception results. Next, the hierarchical feature early fusion module refines the features for predicting planning attributes. Finally, the double-edge interpreter utilizes a late-fusion process specifically designed to integrate lane-level perception and planning information, culminating in the generation of vehicle control signals. Experiments on three Carla benchmarks show significant improvements in driving score of 27.20%, 33.47%, and 15.54% over existing algorithms, respectively, achieving the state-of-the-art performance, with the system operating up to 22.57 FPS.

Related papers

• AlignDrive: Aligned Lateral-Longitudinal Planning for End-to-End Autonomous Driving [23.843979782864427]
  In the planning stage, state-of-the-art autonomous driving models decouple planning into parallel lateral and longitudinal predictions.<n>We propose a novel cascaded framework that explicitly conditions longitudinal planning on the drive path.<n>Our method sets a new SOTA, achieving a driving score of 89.07 and a success rate of 73.18%, demonstrating significantly improved coordination and safety.
  arXiv  Detail & Related papers  (2026-01-05T03:41:20Z)
• Perception in Plan: Coupled Perception and Planning for End-to-End Autonomous Driving [13.367058484125787]
  VeteranAD is a coupled perception and planning framework for end-to-end autonomous driving.<n>We introduce a perception-in-plan framework design, which integrates perception into the planning process.<n>VeteranAD fully unleashes the potential of planning-oriented end-to-end methods, leading to more accurate and reliable driving behavior.
  arXiv  Detail & Related papers  (2025-08-15T14:05:57Z)
• DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Driving [62.62464518137153]
  DriveTransformer is a simplified E2E-AD framework for the ease of scaling up. It is composed of three unified operations: task self-attention, sensor cross-attention, temporal cross-attention. It achieves state-of-the-art performance in both simulated closed-loop benchmark Bench2Drive and real world open-loop benchmark nuScenes with high FPS.
  arXiv  Detail & Related papers  (2025-03-07T11:41:18Z)
• DiFSD: Ego-Centric Fully Sparse Paradigm with Uncertainty Denoising and Iterative Refinement for Efficient End-to-End Self-Driving [55.53171248839489]
  We propose an ego-centric fully sparse paradigm, named DiFSD, for end-to-end self-driving. Specifically, DiFSD mainly consists of sparse perception, hierarchical interaction and iterative motion planner. Experiments conducted on nuScenes and Bench2Drive datasets demonstrate the superior planning performance and great efficiency of DiFSD.
  arXiv  Detail & Related papers  (2024-09-15T15:55:24Z)
• SparseDrive: End-to-End Autonomous Driving via Sparse Scene Representation [11.011219709863875]
  We propose a new end-to-end autonomous driving paradigm named SparseDrive. SparseDrive consists of a symmetric sparse perception module and a parallel motion planner. For motion prediction and planning, we review the great similarity between these two tasks, leading to a parallel design for motion planner.
  arXiv  Detail & Related papers  (2024-05-30T02:13:56Z)
• Graph-based Topology Reasoning for Driving Scenes [102.35885039110057]
  We present TopoNet, the first end-to-end framework capable of abstracting traffic knowledge beyond conventional perception tasks. We evaluate TopoNet on the challenging scene understanding benchmark, OpenLane-V2.
  arXiv  Detail & Related papers  (2023-04-11T15:23:29Z)
• PlanT: Explainable Planning Transformers via Object-Level Representations [64.93938686101309]
  PlanT is a novel approach for planning in the context of self-driving. PlanT is based on imitation learning with a compact object-level input representation. Our results indicate that PlanT can focus on the most relevant object in the scene, even when this object is geometrically distant.
  arXiv  Detail & Related papers  (2022-10-25T17:59:46Z)
• ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learning [132.20119288212376]
  We propose a spatial-temporal feature learning scheme towards a set of more representative features for perception, prediction and planning tasks simultaneously. To the best of our knowledge, we are the first to systematically investigate each part of an interpretable end-to-end vision-based autonomous driving system.
  arXiv  Detail & Related papers  (2022-07-15T16:57:43Z)
• Cooperative Behavioral Planning for Automated Driving using Graph Neural Networks [0.5801044612920815]
  This work proposes to leverage machine learning algorithms to optimize traffic flow at urban intersections by jointly planning for multiple vehicles. Learning-based behavior planning poses several challenges, demanding for a suited input and output representation as well as large amounts of ground-truth data. We train and evaluate the proposed method in an open-source simulation environment for decision making in automated driving.
  arXiv  Detail & Related papers  (2022-02-23T09:36:15Z)
• Deep Structured Reactive Planning [94.92994828905984]
  We propose a novel data-driven, reactive planning objective for self-driving vehicles. We show that our model outperforms a non-reactive variant in successfully completing highly complex maneuvers.
  arXiv  Detail & Related papers  (2021-01-18T01:43:36Z)
• Extracting the fundamental diagram from aerial footage [6.780998887296333]
  Efficient traffic monitoring is playing a fundamental role in successfully tackling congestion in transportation networks. In this paper we devise an innovative way to obtain the fundamental diagram through aerial footage obtained from drone platforms. The derived methodology consists of 3 phases: vehicle detection, vehicle tracking and traffic state estimation.
  arXiv  Detail & Related papers  (2020-07-07T06:34:50Z)
• The Importance of Prior Knowledge in Precise Multimodal Prediction [71.74884391209955]
  Roads have well defined geometries, topologies, and traffic rules. In this paper we propose to incorporate structured priors as a loss function. We demonstrate the effectiveness of our approach on real-world self-driving datasets.
  arXiv  Detail & Related papers  (2020-06-04T03:56:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.