SWA-SOP: Spatially-aware Window Attention for Semantic Occupancy Prediction in Autonomous Driving

• URL: http://arxiv.org/abs/2506.18785v1
• Date: Mon, 23 Jun 2025 15:54:28 GMT
• Title: SWA-SOP: Spatially-aware Window Attention for Semantic Occupancy Prediction in Autonomous Driving
• Abstract summary: Spatially-aware Window Attention (SWA) is a novel mechanism that incorporates local spatial context into attention.<n>SWA significantly improves scene completion and achieves state-of-the-art results on LiDAR-based SOP benchmarks.<n>We further validate its generality by integrating SWA into a camera-based SOP pipeline.
• Score: 16.320467417627277
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Perception systems in autonomous driving rely on sensors such as LiDAR and cameras to perceive the 3D environment. However, due to occlusions and data sparsity, these sensors often fail to capture complete information. Semantic Occupancy Prediction (SOP) addresses this challenge by inferring both occupancy and semantics of unobserved regions. Existing transformer-based SOP methods lack explicit modeling of spatial structure in attention computation, resulting in limited geometric awareness and poor performance in sparse or occluded areas. To this end, we propose Spatially-aware Window Attention (SWA), a novel mechanism that incorporates local spatial context into attention. SWA significantly improves scene completion and achieves state-of-the-art results on LiDAR-based SOP benchmarks. We further validate its generality by integrating SWA into a camera-based SOP pipeline, where it also yields consistent gains across modalities.

Related papers

• SpaceEra++: A Unified Framework Towards 3D Spatial Reasoning in Video [88.79625979053873]
  We propose a novel framework, SpaceEra, for pre-trained vision-language models (VLMs)<n>We extend the original framework into a comprehensive system, termed SpaceEra++, which spans data construction, model design, training optimization, and prompting inference.<n>In addition, to enhance spatial reasoning, we develop SpaceAlign, which enforces pairwise object constraints by jointly exploiting absolute coordinates and relative spatial relations.
  arXiv  Detail & Related papers  (2026-07-02T06:56:29Z)
• TPS-Drive: Task-Guided Representation Purification for VLM-based Autonomous Driving [4.149502662454591]
  Vision-Language Models (VLMs) provide a promising foundation for autonomous driving planning.<n>We introduce TPS-Drive, a novel framework centered on Task-Guided Representation Purification.<n> TPS-Drive achieves accurate agent spatial state forecasting and reduces collision rates in open-loop nuScenes evaluations.
  arXiv  Detail & Related papers  (2026-05-26T13:56:31Z)
• WeatherOcc3D: VLM-Assisted Adverse Weather Aware 3D Semantic Occupancy Prediction [0.0]
  We propose a VLM-assisted framework to guide multi-sensor integration via linguistic environmental cues.<n>We utilize a parameter-efficient adapter to align weather-specific text embeddings with sensor features, coupled with a gating strategy.<n>This enables the model to dynamically modulate the fusion ratio - prioritizing semantic camera features in clear daylight and shifting to geometric LiDAR priors during rainy nights.
  arXiv  Detail & Related papers  (2026-05-15T16:12:50Z)
• Instantaneous Planning, Control and Safety for Navigation in Unknown Underwater Spaces [0.21108097398435333]
  Navigating autonomous underwater vehicles (AUVs) in unknown environments is significantly challenging.<n>These factors pose challenges in accurate global localization, reliable communication, and obstacle avoidance.<n>We propose an integrated planning and control framework that leverages real time sensor data to dynamically induce closed loop AUV trajectories.
  arXiv  Detail & Related papers  (2026-04-07T01:32:29Z)
• VLMFusionOcc3D: VLM Assisted Multi-Modal 3D Semantic Occupancy Prediction [0.0]
  VLMFusionOcc3D is a robust multimodal framework for dense 3D semantic occupancy prediction in autonomous driving.<n>We introduce Weather-Aware Adaptive Fusion, a dynamic gating mechanism that utilizes vehicle metadata and weather-conditioned prompts to re-weight sensor contributions.<n>Our approach achieves significant improvements in challenging weather scenarios, offering a scalable and robust solution for complex urban navigation.
  arXiv  Detail & Related papers  (2026-03-03T05:22:28Z)
• HiST-VLA: A Hierarchical Spatio-Temporal Vision-Language-Action Model for End-to-End Autonomous Driving [20.266736153749417]
  Vision-Language-Action (VLA) models offer promising capabilities for autonomous driving through multimodal understanding.<n>Their utilization in safety-critical scenarios is constrained by inherent limitations, including numerical reasoning, weak 3D spatial awareness, and high sensitivity to context.<n>We propose HiST-VLA, a novel Hierarchical Spatio-Temporal VLA model designed for reliable trajectory generation.
  arXiv  Detail & Related papers  (2026-02-11T07:08:33Z)
• RSGround-R1: Rethinking Remote Sensing Visual Grounding through Spatial Reasoning [61.84363374647606]
  Remote Sensing Visual Grounding (RSVG) aims to localize target objects in large-scale aerial imagery based on natural language descriptions.<n>These descriptions often rely heavily on positional cues, posing unique challenges for Multimodal Large Language Models (MLLMs) in spatial reasoning.<n>We propose a reasoning-guided, position-aware post-training framework, dubbed textbfRSGround-R1, to progressively enhance spatial understanding.
  arXiv  Detail & Related papers  (2026-01-29T12:35:57Z)
• ZTRS: Zero-Imitation End-to-end Autonomous Driving with Trajectory Scoring [52.195295396336526]
  ZTRS (Zero-Imitation End-to-End Autonomous Driving with Trajectory Scoring) is a framework that combines the strengths of both worlds: sensor inputs without losing information and RL training for robust planning.<n>ZTRS demonstrates strong performance across three benchmarks: Navtest, Navhard, and HUGSIM.
  arXiv  Detail & Related papers  (2025-10-28T06:26:36Z)
• Backscatter Device-aided Integrated Sensing and Communication: A Pareto Optimization Framework [59.30060797118097]
  Integrated sensing and communication (ISAC) systems potentially encounter significant performance degradation in densely obstructed urban non-line-of-sight scenarios.<n>This paper proposes a backscatter approximation (BD)-assisted ISAC system, which leverages passive BDs naturally distributed in environments of enhancement.
  arXiv  Detail & Related papers  (2025-07-12T17:11:06Z)
• A Synthetic Benchmark for Collaborative 3D Semantic Occupancy Prediction in V2X Autonomous Driving [3.6538681992157604]
  3D semantic occupancy prediction is an emerging perception paradigm in autonomous driving.<n>We augment an existing collaborative perception dataset by replaying it in CARLA with a high-resolution semantic voxel sensor.<n>We develop a baseline model that performs inter-agent feature fusion via spatial alignment and attention aggregation.
  arXiv  Detail & Related papers  (2025-06-20T13:58:10Z)
• DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Driving [62.62464518137153]
  DriveTransformer is a simplified E2E-AD framework for the ease of scaling up.<n>It is composed of three unified operations: task self-attention, sensor cross-attention, temporal cross-attention.<n>It achieves state-of-the-art performance in both simulated closed-loop benchmark Bench2Drive and real world open-loop benchmark nuScenes with high FPS.
  arXiv  Detail & Related papers  (2025-03-07T11:41:18Z)
• EDENet: Echo Direction Encoding Network for Place Recognition Based on Ground Penetrating Radar [20.860547422960796]
  Ground penetrating radar (GPR) based localization has gained significant recognition in robotics.<n>Existing methods are primarily focused on small-scale place recognition (PR), leaving the challenges of PR in large-scale maps unaddressed.<n>We introduce learnable Gabor filters for the precise extraction of directional responses, coupled with a direction-aware attention mechanism for effective geometric encoding.
  arXiv  Detail & Related papers  (2025-02-28T01:48:12Z)
• DiffSSC: Semantic LiDAR Scan Completion using Denoising Diffusion Probabilistic Models [18.342569823885864]
  3D LiDAR sensors are widely used to capture sparse point clouds of the vehicle's surroundings. Such systems struggle to perceive occluded areas and gaps in the scene due to the sparsity of these point clouds and their lack of semantics. We jointly predict unobserved geometry and semantics in the scene given raw LiDAR measurements, aiming for a more complete scene representation.
  arXiv  Detail & Related papers  (2024-09-26T17:39:05Z)
• OOSTraj: Out-of-Sight Trajectory Prediction With Vision-Positioning Denoising [49.86409475232849]
  Trajectory prediction is fundamental in computer vision and autonomous driving. Existing approaches in this field often assume precise and complete observational data. We present a novel method for out-of-sight trajectory prediction that leverages a vision-positioning technique.
  arXiv  Detail & Related papers  (2024-04-02T18:30:29Z)
• Efficient Real-time Smoke Filtration with 3D LiDAR for Search and Rescue with Autonomous Heterogeneous Robotic Systems [56.838297900091426]
  Smoke and dust affect the performance of any mobile robotic platform due to their reliance on onboard perception systems. This paper proposes a novel modular computation filtration pipeline based on intensity and spatial information.
  arXiv  Detail & Related papers  (2023-08-14T16:48:57Z)
• OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perception [73.05425657479704]
  We propose OpenOccupancy, which is the first surrounding semantic occupancy perception benchmark. We extend the large-scale nuScenes dataset with dense semantic occupancy annotations. Considering the complexity of surrounding occupancy perception, we propose the Cascade Occupancy Network (CONet) to refine the coarse prediction.
  arXiv  Detail & Related papers  (2023-03-07T15:43:39Z)
• USC: Uncompromising Spatial Constraints for Safety-Oriented 3D Object Detectors in Autonomous Driving [7.355977594790584]
  We consider the safety-oriented performance of 3D object detectors in autonomous driving contexts.<n>We present uncompromising spatial constraints (USC), which characterize a simple yet important localization requirement.<n>We incorporate the quantitative measures into common loss functions to enable safety-oriented fine-tuning for existing models.
  arXiv  Detail & Related papers  (2022-09-21T14:03:08Z)
• Large-scale Autonomous Flight with Real-time Semantic SLAM under Dense Forest Canopy [48.51396198176273]
  We propose an integrated system that can perform large-scale autonomous flights and real-time semantic mapping in challenging under-canopy environments. We detect and model tree trunks and ground planes from LiDAR data, which are associated across scans and used to constrain robot poses as well as tree trunk models. A drift-compensation mechanism is designed to minimize the odometry drift using semantic SLAM outputs in real time, while maintaining planner optimality and controller stability.
  arXiv  Detail & Related papers  (2021-09-14T07:24:53Z)
• Towards robust sensing for Autonomous Vehicles: An adversarial perspective [82.83630604517249]
  It is of primary importance that the resulting decisions are robust to perturbations. Adversarial perturbations are purposefully crafted alterations of the environment or of the sensory measurements. A careful evaluation of the vulnerabilities of their sensing system(s) is necessary in order to build and deploy safer systems.
  arXiv  Detail & Related papers  (2020-07-14T05:25:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.