OccSTeP: Benchmarking 4D Occupancy Spatio-Temporal Persistence

• URL: http://arxiv.org/abs/2512.15621v1
• Date: Wed, 17 Dec 2025 17:29:20 GMT
• Title: OccSTeP: Benchmarking 4D Occupancy Spatio-Temporal Persistence
• Abstract summary: We introduce a new concept of 4D Occupancy Spatio-Temporal Persistence (OccSTeP)<n>This concept aims to address two tasks: (1) reactive forecasting: ''what happen'' and (2) proactive forecasting: "what would happen given a specific future action"<n>We propose OccSTeP-WM, a tokenizer-free world model that maintains a dense voxel-based scene state and fuses incrementally-temporal context over time.
• Score: 19.81329748424203
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Autonomous driving requires a persistent understanding of 3D scenes that is robust to temporal disturbances and accounts for potential future actions. We introduce a new concept of 4D Occupancy Spatio-Temporal Persistence (OccSTeP), which aims to address two tasks: (1) reactive forecasting: ''what will happen next'' and (2) proactive forecasting: "what would happen given a specific future action". For the first time, we create a new OccSTeP benchmark with challenging scenarios (e.g., erroneous semantic labels and dropped frames). To address this task, we propose OccSTeP-WM, a tokenizer-free world model that maintains a dense voxel-based scene state and incrementally fuses spatio-temporal context over time. OccSTeP-WM leverages a linear-complexity attention backbone and a recurrent state-space module to capture long-range spatial dependencies while continually updating the scene memory with ego-motion compensation. This design enables online inference and robust performance even when historical sensor input is missing or noisy. Extensive experiments prove the effectiveness of the OccSTeP concept and our OccSTeP-WM, yielding an average semantic mIoU of 23.70% (+6.56% gain) and occupancy IoU of 35.89% (+9.26% gain). The data and code will be open source at https://github.com/FaterYU/OccSTeP.

Related papers

• From Where to How: Continuous 4D Interaction Forecasting from Egocentric Video [62.898774619819676]
  Egocentric 4D interaction forecasting aims to anticipate both where future interactions will occur in 3D and how the human body will move to realize them.<n>We introduce Coherent4D, a large-scale egocentric dataset for continuous 4D interaction forecasting, comprising approximately 233K samples across three domains.<n>We propose HIGFlow, a Hand Interaction Guided Residual Flow framework that models forecasting as a cascaded where-to-how process.
  arXiv  Detail & Related papers  (2026-09-08T12:06:23Z)
• InterOCF: Spatio-Temporal 2D-3D Interaction for Camera-Only 4D Occupancy Forecasting [14.13719947869133]
  Camera-only 4D occupancy forecasting enables autonomous vehicles to predict future 3D semantic scenes solely from historical multi-view images.<n>We introduce a novel framework, InterOCF, for 4D occupancy forecasting that jointly models temporal dynamics in both 3D voxel-based representations and multi-view segmentation sequences.<n>Our framework incorporates three core components: 1) A 3D Spatio-Temporal (3DST) module that learns volumetric dynamics from historical voxel states to predict future voxel states; 2) A 2D Spatio-Temporal (2DST) module employing an auxiliary multi-view temporal segmentation forecasting
  arXiv  Detail & Related papers  (2026-07-27T13:43:37Z)
• E-3DPSM: A State Machine for Event-Based Egocentric 3D Human Pose Estimation [73.63705002652638]
  Event cameras offer advantages in monocular egocentric 3D human pose estimation from head-mounted devices.<n> Existing methods suffer from low 3D estimation accuracy, insufficient in many applications.<n>This paper introduces E-3DPSM, an event-driven continuous pose state machine for event-based egocentric 3D human pose estimation.
  arXiv  Detail & Related papers  (2026-04-09T17:59:52Z)
• RoboStream: Weaving Spatio-Temporal Reasoning with Memory in Vision-Language Models for Robotics [25.944338247511215]
  VLM-based planners treat each step as an observation-to-action mapping, forcing them to reinfer scene geometry from raw pixels at every decision point.<n>We propose SStream-Temporal Fusion Tokens (TF-Tokens), which bind visual evidence to 3D geometric attributes for persistent object grounding.<n>This design enables the planner to trace causal chains and preserve object permanence under occlusion without additional training or fine-tuning.
  arXiv  Detail & Related papers  (2026-03-13T12:34:26Z)
• ForecastOcc: Vision-based Semantic Occupancy Forecasting [16.699381591572163]
  We present ForecastOcc, the first framework for vision-based semantic occupancy forecasting that predicts future occupancy states and semantic categories.<n>Our framework yields semantic occupancy forecasts for multiple horizons directly from past camera images, without relying on externally estimated maps.
  arXiv  Detail & Related papers  (2026-02-08T15:16:06Z)
• Occupancy Learning with Spatiotemporal Memory [39.41175479685905]
  We propose a scene-level occupancy representation learning framework that effectively learns 3D occupancy feature with temporal consistency.<n>Our method significantly enhances thetemporal representation learned for 3D occupancy prediction tasks by exploiting the temporal dependency between multi-frame inputs.
  arXiv  Detail & Related papers  (2025-08-06T17:59:52Z)
• GaussianWorld: Gaussian World Model for Streaming 3D Occupancy Prediction [67.81475355852997]
  3D occupancy prediction is important for autonomous driving due to its comprehensive perception of the surroundings.<n>We propose a world-model-based framework to exploit the scene evolution for perception.<n>Our framework improves the performance of the single-frame counterpart by over 2% in mIoU without introducing additional computations.
  arXiv  Detail & Related papers  (2024-12-13T18:59:54Z)
• Hierarchical Temporal Context Learning for Camera-based Semantic Scene Completion [57.232688209606515]
  We present HTCL, a novel Temporal Temporal Context Learning paradigm for improving camera-based semantic scene completion. Our method ranks $1st$ on the Semantic KITTI benchmark and even surpasses LiDAR-based methods in terms of mIoU.
  arXiv  Detail & Related papers  (2024-07-02T09:11:17Z)
• DriveWorld: 4D Pre-trained Scene Understanding via World Models for Autonomous Driving [67.46481099962088]
  Current vision-centric pre-training typically relies on either 2D or 3D pre-text tasks, overlooking the temporal characteristics of autonomous driving as a 4D scene understanding task. We introduce emphcentricDriveWorld, which is capable of pre-training from multi-camera driving videos in atemporal fashion. DriveWorld delivers promising results on various autonomous driving tasks.
  arXiv  Detail & Related papers  (2024-05-07T15:14:20Z)
• MLP: Motion Label Prior for Temporal Sentence Localization in Untrimmed 3D Human Motions [20.986063755422173]
  We aim to locate a target moment from a 3D human motion that semantically corresponds to a text query. To refine this, we devise two novel label-prior-knowledge training schemes. We show that injecting label-prior knowledge into the model is crucial for improving performance sequences at high IoU.
  arXiv  Detail & Related papers  (2024-04-21T13:25:46Z)
• Fully Sparse 3D Occupancy Prediction [37.265473869812816]
  Occupancy prediction plays a pivotal role in autonomous driving. Previous methods typically construct dense 3D volumes, neglecting the inherent sparsity of the scene and suffering from high computational costs. We introduce a novel fully sparse occupancy network, termed SparseOcc. SparseOcc initially reconstructs a sparse 3D representation from camera-only inputs and subsequently predicts semantic/instance occupancy from the 3D sparse representation by sparse queries.
  arXiv  Detail & Related papers  (2023-12-28T16:54:53Z)
• Regularizing Self-supervised 3D Scene Flows with Surface Awareness and Cyclic Consistency [3.124750429062221]
  We introduce two new consistency losses that enlarge clusters while preventing them from spreading over distinct objects. The proposed losses are model-independent and can thus be used in a plug-and-play fashion to significantly improve the performance of existing models. We also showcase the effectiveness and generalization capability of our framework on four standard sensor-unique driving datasets.
  arXiv  Detail & Related papers  (2023-12-12T11:00:39Z)
• Nothing Stands Still: A Spatiotemporal Benchmark on 3D Point Cloud Registration Under Large Geometric and Temporal Change [82.31647863785923]
  Building 3D geometric maps of man-made spaces are fundamental computer vision and robotics.<n>Nothing Stands Still (NSS) benchmark focuses on thetemporal registration of 3D scenes undergoing large spatial and temporal change.<n>As part of NSS, we introduce a dataset of 3D point clouds recurrently captured in large-scale building indoor environments that are under construction or renovation.
  arXiv  Detail & Related papers  (2023-11-15T20:09:29Z)
• Implicit Occupancy Flow Fields for Perception and Prediction in Self-Driving [68.95178518732965]
  A self-driving vehicle (SDV) must be able to perceive its surroundings and predict the future behavior of other traffic participants. Existing works either perform object detection followed by trajectory of the detected objects, or predict dense occupancy and flow grids for the whole scene. This motivates our unified approach to perception and future prediction that implicitly represents occupancy and flow over time with a single neural network.
  arXiv  Detail & Related papers  (2023-08-02T23:39:24Z)
• P-STMO: Pre-Trained Spatial Temporal Many-to-One Model for 3D Human Pose Estimation [78.83305967085413]
  This paper introduces a novel Pre-trained Spatial Temporal Many-to-One (P-STMO) model for 2D-to-3D human pose estimation task. Our method outperforms state-of-the-art methods with fewer parameters and less computational overhead.
  arXiv  Detail & Related papers  (2022-03-15T04:00:59Z)
• A Spatio-temporal Transformer for 3D Human Motion Prediction [39.31212055504893]
  We propose a Transformer-based architecture for the task of generative modelling of 3D human motion. We empirically show that this effectively learns the underlying motion dynamics and reduces error accumulation over time observed in auto-gressive models.
  arXiv  Detail & Related papers  (2020-04-18T19:49:28Z)
• A Graph Attention Spatio-temporal Convolutional Network for 3D Human Pose Estimation in Video [7.647599484103065]
  We improve the learning of constraints in human skeleton by modeling local global spatial information via attention mechanisms. Our approach effectively mitigates depth ambiguity and self-occlusion, generalizes to half upper body estimation, and achieves competitive performance on 2D-to-3D video pose estimation.
  arXiv  Detail & Related papers  (2020-03-11T14:54:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.