Plug-and-Play Traffic Element Awareness for End-to-End Autonomous Driving

• URL: http://arxiv.org/abs/2608.18035v1
• Date: Tue, 18 Aug 2026 17:29:51 GMT
• Title: Plug-and-Play Traffic Element Awareness for End-to-End Autonomous Driving
• Abstract summary: We present the first systematic investigation of traffic element awareness for end-to-end autonomous driving.<n>We construct a unified research infrastructure by augmenting multiple public driving datasets with traffic-element annotations.<n>We evaluate this design across modern paradigms, including perception-prediction-planning pipelines, vision-language-action models (VLA), regression-based planners, diffusion-based policies, and trajectory-scoring frameworks.
• Score: 22.38391140804806
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Traffic elements such as traffic lights and road signs play a fundamental role in human driving decisions and should naturally influence end-to-end driving performance. However, existing end-to-end driving research predominantly focuses on dynamic road participants (e.g., vehicles and pedestrians), while the role of traffic elements remains largely unexplored. The community still lacks a systematic study quantifying their impact, largely because public datasets rarely provide structured traffic-element annotations and modern driving systems vary widely in architecture and training paradigm. In this work, we present the first systematic investigation of traffic element awareness for end-to-end autonomous driving. We construct a unified research infrastructure by augmenting multiple public driving datasets with comprehensive traffic-element annotations. To support diverse model families, we adopt a minimal and universal integration design that incorporates traffic-element signals into existing pipelines in a plug-and-play manner with negligible architectural modification. We evaluate this design across modern paradigms, including perception-prediction-planning pipelines, vision-language-action models (VLA), regression-based planners, diffusion-based policies, and trajectory-scoring frameworks, on nuScenes, NAVSIM-v1, NAVSIM-v2, and Bench2Drive. Across all paradigms and datasets, this simple integration consistently improves driving performance, demonstrating that traffic element awareness provides a robust and generalizable signal for end-to-end driving systems. Notably, on the challenging NAVSIM-v2 benchmark, our approach significantly improves state-of-the-art architectures and data pipelines, establishing a new state of the art.

Related papers

• Learning from the Unseen: Generative Data Augmentation for Geometric-Semantic Accident Anticipation [22.129589116442208]
  Anticipating traffic accidents is a critical yet unresolved problem for autonomous driving.<n>We propose a dual-path framework that produces high-fidelity synthetic driving scenes consistent with the statistical patterns of real data.<n>We also design a graph neural network enriched with semantic cues, enabling dynamic reasoning over both spatial and semantic relations.
  arXiv  Detail & Related papers  (2026-04-29T15:29:19Z)
• Infrastructure-Centric World Models: Bridging Temporal Depth and Spatial Breadth for Roadside Perception [3.3242611619309614]
  This paper presents a vision for Infrastructure-centric World Models (I-WM) in three phases.<n>We propose a dual-layer architecture, annotation-free perception as a multi-modal data engine feeding end-to-end generative world models.<n>We establish a taxonomy of driving world model paradigms, position I-WM relative to LeCun's JEPA, Li Fei-Fei's spatial intelligence, and VLA architectures.
  arXiv  Detail & Related papers  (2026-04-19T22:50:32Z)
• HetroD: A High-Fidelity Drone Dataset and Benchmark for Autonomous Driving in Heterogeneous Traffic [49.31491001465465]
  HetroD is a dataset and benchmark for developing autonomous driving systems in heterogeneous environments.<n>HetroD targets the critical challenge of navi- gating real-world heterogeneous traffic dominated by vulner- able road users (VRUs)
  arXiv  Detail & Related papers  (2026-02-03T12:12:47Z)
• Contrastive Learning-Driven Traffic Sign Perception: Multi-Modal Fusion of Text and Vision [2.0720154517628417]
  We propose a novel framework combining open-vocabulary detection and cross-modal learning.<n>For traffic sign detection, our NanoVerse YOLO model integrates a vision-language path aggregation network (RepVL-PAN) and an SPD-Conv module.<n>For traffic sign classification, we designed a Traffic Sign Recognition Multimodal Contrastive Learning model (TSR-MCL)<n>On the TT100K dataset, our method achieves a state-of-the-art 78.4% mAP in the long-tail detection task for all-class recognition.
  arXiv  Detail & Related papers  (2025-07-31T08:23:30Z)
• LANet: A Lane Boundaries-Aware Approach For Robust Trajectory Prediction [4.096453902709292]
  We propose an enhanced motion forecasting model informed by multiple vector map elements, including lane boundaries and road edges.<n>An effective feature fusion strategy is developed to merge information in different vector map components, where the model learns holistic information on road structures.<n>Our method provides a more informative and efficient representation of the driving environment and advances the state of the art for autonomous vehicle motion forecasting.
  arXiv  Detail & Related papers  (2025-07-02T02:49:24Z)
• Towards Intelligent Transportation with Pedestrians and Vehicles In-the-Loop: A Surveillance Video-Assisted Federated Digital Twin Framework [62.47416496137193]
  We propose a surveillance video assisted federated digital twin (SV-FDT) framework to empower ITSs with pedestrians and vehicles in-the-loop.<n>The architecture consists of three layers: (i) the end layer, which collects traffic surveillance videos from multiple sources; (ii) the edge layer, responsible for semantic segmentation-based visual understanding, twin agent-based interaction modeling, and local digital twin system (LDTS) creation in local regions; and (iii) the cloud layer, which integrates LDTSs across different regions to construct a global DT model in realtime.
  arXiv  Detail & Related papers  (2025-03-06T07:36:06Z)
• Knowledge-Informed Multi-Agent Trajectory Prediction at Signalized Intersections for Infrastructure-to-Everything [7.452533291998081]
  We introduce an Infrastructure-to-Everything (I2X) collaborative prediction scheme.<n>In this scheme, roadside units (RSUs) independently forecast the future trajectories of all vehicles.<n>We propose I2XTraj, a dedicated infrastructure-based trajectory prediction model.
  arXiv  Detail & Related papers  (2025-01-23T08:23:45Z)
• High-Order Evolving Graphs for Enhanced Representation of Traffic Dynamics [4.856478623606766]
  We present an innovative framework for traffic dynamics analysis using High-Order Evolving Graphs, designed to improve representations in autonomous driving. Our approach constructs bidirectional temporal bipartite graphs that effectively model the complex interactions within traffic scenes in real-time.
  arXiv  Detail & Related papers  (2024-09-17T14:00:58Z)
• Leveraging Driver Field-of-View for Multimodal Ego-Trajectory Prediction [69.29802752614677]
  RouteFormer is a novel ego-trajectory prediction network combining GPS data, environmental context, and the driver's field-of-view.<n>To tackle data scarcity and enhance diversity, we introduce GEM, a dataset of urban driving scenarios enriched with synchronized driver field-of-view and gaze data.
  arXiv  Detail & Related papers  (2023-12-13T23:06:30Z)
• OpenLane-V2: A Topology Reasoning Benchmark for Unified 3D HD Mapping [84.65114565766596]
  We present OpenLane-V2, the first dataset on topology reasoning for traffic scene structure. OpenLane-V2 consists of 2,000 annotated road scenes that describe traffic elements and their correlation to the lanes. We evaluate various state-of-the-art methods, and present their quantitative and qualitative results on OpenLane-V2 to indicate future avenues for investigating topology reasoning in traffic scenes.
  arXiv  Detail & Related papers  (2023-04-20T16:31:22Z)
• Road Network Guided Fine-Grained Urban Traffic Flow Inference [108.64631590347352]
  Accurate inference of fine-grained traffic flow from coarse-grained one is an emerging yet crucial problem. We propose a novel Road-Aware Traffic Flow Magnifier (RATFM) that exploits the prior knowledge of road networks. Our method can generate high-quality fine-grained traffic flow maps.
  arXiv  Detail & Related papers  (2021-09-29T07:51:49Z)
• End-to-End Intersection Handling using Multi-Agent Deep Reinforcement Learning [63.56464608571663]
  Navigating through intersections is one of the main challenging tasks for an autonomous vehicle. In this work, we focus on the implementation of a system able to navigate through intersections where only traffic signs are provided. We propose a multi-agent system using a continuous, model-free Deep Reinforcement Learning algorithm used to train a neural network for predicting both the acceleration and the steering angle at each time step.
  arXiv  Detail & Related papers  (2021-04-28T07:54:40Z)
• Multi-Modal Fusion Transformer for End-to-End Autonomous Driving [59.60483620730437]
  We propose TransFuser, a novel Multi-Modal Fusion Transformer, to integrate image and LiDAR representations using attention. Our approach achieves state-of-the-art driving performance while reducing collisions by 76% compared to geometry-based fusion.
  arXiv  Detail & Related papers  (2021-04-19T11:48:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.