Multi-Modal Traffic Sign Detection with Semantic Attributes for Autonomous Driving

• URL: http://arxiv.org/abs/2608.20874v1
• Date: Fri, 21 Aug 2026 08:51:39 GMT
• Title: Multi-Modal Traffic Sign Detection with Semantic Attributes for Autonomous Driving
• Abstract summary: Vision-based methods face three fundamental limitations: poor cross-regional generalization due to high diversity across countries, degraded performance on small-object detection at long ranges, and fragile temporal tracking under strongly non-linear perspective distortion that occurs as a vehicle approaches a sign.<n>We present a multi-modal detection framework whose Intensity-Aware Deformable Fusion module aligns retro-reflective LiDAR cues with camera features, anchoring detection on geometric invariants rather than region-specific visual appearance.<n>Extensive evaluation on our dataset, spanning 60+ countries and 2,500+ hours of driving data, shows that the proposed pipeline achieves an Object Miss Ratio (
• Score: 36.63134402686799
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reliable traffic sign detection is a prerequisite for the global deployment of autonomous driving systems, where regulatory compliance and road safety depend on perceiving signs correctly across regions, ranges, and weather conditions. Despite recent progress, vision-based methods continue to face three fundamental limitations: poor cross-regional generalization due to high diversity across countries, degraded performance on small-object detection at long ranges (traffic signs occupy as little as $10{\times}10$ pixels at 200m), and fragile temporal tracking under the strongly non-linear perspective distortion that occurs as a vehicle approaches a sign. In this paper, we address the problem of robust, long-range, region-agnostic traffic sign perception by combining camera and Light Detection and Ranging (LiDAR) sensing. We present a multi-modal detection framework whose Intensity-Aware Deformable Fusion module aligns retro-reflective LiDAR cues with camera features, anchoring detection on geometric invariants rather than region-specific visual appearance. We further introduce a dual motion-model tracker that explicitly accounts for non-linear perspective transformations during vehicle approach, substantially improving temporal consistency over linear motion assumptions. Additionally, we develop a semantic attribute classification pipeline that estimates occlusion level, readability, sign embeddedness, and road relevance, providing actionable context to downstream planning. Extensive evaluation on our dataset, spanning 60+ countries and 2,500+ hours of driving data, shows that the proposed pipeline achieves an Object Miss Ratio (OMR) of 0.49% across 221,068 evaluation sequences, demonstrating globally generalizable traffic sign perception in commercial-grade autonomous driving systems.

Related papers

• Optimization-Guided Diffusion for Interactive Scene Generation [52.23368750264419]
  We present OMEGA, an optimization-guided, training-free framework that enforces structural consistency and interaction awareness during diffusion-based sampling.<n>We show that OMEGA improves generation realism, consistency, and controllability, increasing the ratio of physically and behaviorally valid scenes.<n>Our approach can also generate $5times$ more near-collision frames with a time-to-collision under three seconds.
  arXiv  Detail & Related papers  (2025-12-08T15:56:18Z)
• Navigating in the Dark: A Multimodal Framework and Dataset for Nighttime Traffic Sign Recognition [9.151417330446591]
  INTSD is a large-scale dataset of street-level night-time images of traffic signboards collected across diverse regions of India.<n>We propose LENS-Net, which integrates an adaptive image enhancement detector for joint illumination correction and sign localization.<n>Our method surpasses existing frameworks, with ablation studies confirming the effectiveness of its key components.
  arXiv  Detail & Related papers  (2025-11-21T12:04:53Z)
• UAV-Based Intelligent Traffic Surveillance System: Real-Time Vehicle Detection, Classification, Tracking, and Behavioral Analysis [2.6341839987582345]
  This paper introduces an advanced unmanned aerial vehicle (UAV)-based traffic surveillance system.<n>It is capable of accurate vehicle detection, classification, tracking, and behavioral analysis in real-world, unconstrained urban environments.
  arXiv  Detail & Related papers  (2025-09-04T19:19:25Z)
• Lane-Wise Highway Anomaly Detection [8.086502588472783]
  This paper proposes a scalable and interpretable framework for lane-wise highway traffic anomaly detection.<n>Unlike traditional sensor-dependent methods, our approach uses AI-powered vision models to extract lane-specific features.<n>Our framework outperforms state-of-the-art methods in precision, recall, and F1-score.
  arXiv  Detail & Related papers  (2025-05-05T12:32:23Z)
• Reinforcement Learning with Latent State Inference for Autonomous On-ramp Merging under Observation Delay [6.0111084468944]
  We introduce the Lane-keeping, Lane-changing with Latent-state Inference and Safety Controller (L3IS) agent. L3IS is designed to perform the on-ramp merging task safely without comprehensive knowledge about surrounding vehicles' intents or driving styles. We present an augmentation of this agent called AL3IS that accounts for observation delays, allowing the agent to make more robust decisions in real-world environments.
  arXiv  Detail & Related papers  (2024-03-18T15:02:46Z)
• Leveraging Driver Field-of-View for Multimodal Ego-Trajectory Prediction [69.29802752614677]
  RouteFormer is a novel ego-trajectory prediction network combining GPS data, environmental context, and the driver's field-of-view.<n>To tackle data scarcity and enhance diversity, we introduce GEM, a dataset of urban driving scenarios enriched with synchronized driver field-of-view and gaze data.
  arXiv  Detail & Related papers  (2023-12-13T23:06:30Z)
• Unsupervised Domain Adaptation for Self-Driving from Past Traversal Features [69.47588461101925]
  We propose a method to adapt 3D object detectors to new driving environments. Our approach enhances LiDAR-based detection models using spatial quantized historical features. Experiments on real-world datasets demonstrate significant improvements.
  arXiv  Detail & Related papers  (2023-09-21T15:00:31Z)
• Unsupervised Adaptation from Repeated Traversals for Autonomous Driving [54.59577283226982]
  Self-driving cars must generalize to the end-user's environment to operate reliably. One potential solution is to leverage unlabeled data collected from the end-users' environments. There is no reliable signal in the target domain to supervise the adaptation process. We show that this simple additional assumption is sufficient to obtain a potent signal that allows us to perform iterative self-training of 3D object detectors on the target domain.
  arXiv  Detail & Related papers  (2023-03-27T15:07:55Z)
• Multi-Modal Fusion Transformer for End-to-End Autonomous Driving [59.60483620730437]
  We propose TransFuser, a novel Multi-Modal Fusion Transformer, to integrate image and LiDAR representations using attention. Our approach achieves state-of-the-art driving performance while reducing collisions by 76% compared to geometry-based fusion.
  arXiv  Detail & Related papers  (2021-04-19T11:48:13Z)
• Phase Space Reconstruction Network for Lane Intrusion Action Recognition [9.351931162958465]
  In this paper, we propose a novel object-level phase space reconstruction network (PSRNet) for motion time series classification. Our PSRNet could reach the best accuracy of 98.0%, which remarkably exceeds existing action recognition approaches by more than 30%.
  arXiv  Detail & Related papers  (2021-02-22T16:18:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.