OccTENS: 3D Occupancy World Model via Temporal Next-Scale Prediction

• URL: http://arxiv.org/abs/2509.03887v1
• Date: Thu, 04 Sep 2025 05:06:47 GMT
• Title: OccTENS: 3D Occupancy World Model via Temporal Next-Scale Prediction
• Authors: Bu Jin, Songen Gu, Xiaotao Hu, Yupeng Zheng, Xiaoyang Guo, Qian Zhang, Xiaoxiao Long, Wei Yin,
• Abstract summary: OccTENS is a generative occupancy world model that enables controllable, high-fidelity long-term occupancy generation.<n>We reformulate the occupancy world model as a temporal next-scale prediction (TENS) task.<n>OccTENS outperforms the state-of-the-art method with both higher occupancy quality and faster inference time.
• Score: 25.630452373274636
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In this paper, we propose OccTENS, a generative occupancy world model that enables controllable, high-fidelity long-term occupancy generation while maintaining computational efficiency. Different from visual generation, the occupancy world model must capture the fine-grained 3D geometry and dynamic evolution of the 3D scenes, posing great challenges for the generative models. Recent approaches based on autoregression (AR) have demonstrated the potential to predict vehicle movement and future occupancy scenes simultaneously from historical observations, but they typically suffer from \textbf{inefficiency}, \textbf{temporal degradation} in long-term generation and \textbf{lack of controllability}. To holistically address these issues, we reformulate the occupancy world model as a temporal next-scale prediction (TENS) task, which decomposes the temporal sequence modeling problem into the modeling of spatial scale-by-scale generation and temporal scene-by-scene prediction. With a \textbf{TensFormer}, OccTENS can effectively manage the temporal causality and spatial relationships of occupancy sequences in a flexible and scalable way. To enhance the pose controllability, we further propose a holistic pose aggregation strategy, which features a unified sequence modeling for occupancy and ego-motion. Experiments show that OccTENS outperforms the state-of-the-art method with both higher occupancy quality and faster inference time.

Related papers

• Scalable Spatio-Temporal SE(3) Diffusion for Long-Horizon Protein Dynamics [51.85385061275941]
  Molecular dynamics (MD) simulations remain the gold standard for studying protein dynamics.<n>Recent generative models have shown promise in accelerating simulations, yet they struggle with long-horizon generation.<n>We present STAR-MD, a scalable diffusion model that generates physically plausible protein trajectories over micro-scale timescales.
  arXiv  Detail & Related papers  (2026-02-02T14:13:28Z)
• SAMPO:Scale-wise Autoregression with Motion PrOmpt for generative world models [42.814012901180774]
  textbfSAMPO is a hybrid framework that combines visual autoregressive modeling for intra-frame generation with causal modeling for next-frame generation.<n>We show that SAMPO achieves competitive performance in action-conditioned video prediction and model-based control.<n>We also evaluate SAMPO's zero-shot generalization and scaling behavior, demonstrating its ability to generalize to unseen tasks.
  arXiv  Detail & Related papers  (2025-09-19T02:41:37Z)
• StarPose: 3D Human Pose Estimation via Spatial-Temporal Autoregressive Diffusion [29.682018018059043]
  StarPose is an autoregressive diffusion framework for 3D human pose estimation.<n>It incorporates historical 3D pose predictions and spatial-temporal physical guidance.<n>It achieves superior accuracy and temporal consistency in 3D human pose estimation.
  arXiv  Detail & Related papers  (2025-08-04T04:50:05Z)
• Next-Generation Conflict Forecasting: Unleashing Predictive Patterns through Spatiotemporal Learning [0.0]
  This study presents a neural network architecture for forecasting three distinct types of violence up to 36 months in advance.<n>The model jointly performs probabilistic classification and regression tasks, producing both estimates and expected magnitudes of future events.<n>It is a promising tool for warning systems, humanitarian response planning, and evidence-based peacebuilding initiatives.
  arXiv  Detail & Related papers  (2025-06-08T20:42:29Z)
• Learning 3D Persistent Embodied World Models [84.40585374179037]
  We introduce a new persistent embodied world model with an explicit memory of previously generated content.<n>During generation time, our video diffusion model predicts RGB-D video of the future observations of the agent.<n>This generation is then aggregated into a persistent 3D map of the environment.
  arXiv  Detail & Related papers  (2025-05-05T17:59:17Z)
• An Efficient Occupancy World Model via Decoupled Dynamic Flow and Image-assisted Training [50.71892161377806]
  DFIT-OccWorld is an efficient 3D occupancy world model that leverages decoupled dynamic flow and image-assisted training strategy.<n>Our model forecasts future dynamic voxels by warping existing observations using voxel flow, whereas static voxels are easily obtained through pose transformation.
  arXiv  Detail & Related papers  (2024-12-18T12:10:33Z)
• DOME: Taming Diffusion Model into High-Fidelity Controllable Occupancy World Model [14.996395953240699]
  DOME is a diffusion-based world model that predicts future occupancy frames based on past occupancy observations. The ability of this world model to capture the evolution of the environment is crucial for planning in autonomous driving.
  arXiv  Detail & Related papers  (2024-10-14T12:24:32Z)
• STGFormer: Spatio-Temporal GraphFormer for 3D Human Pose Estimation in Video [7.345621536750547]
  This paper presents the S-Temporal GraphFormer framework (STGFormer) for 3D human pose estimation in videos.<n>First, we introduce a STG attention mechanism, designed to more effectively leverage the inherent graph distributions of human body.<n>Next, we present a Modulated Hop-wise Regular GCN to independently process temporal and spatial dimensions in parallel.<n>Finally, we demonstrate our method state-of-the-art performance on the Human3.6M and MPIINF-3DHP datasets.
  arXiv  Detail & Related papers  (2024-07-14T06:45:27Z)
• Generative Modeling with Phase Stochastic Bridges [49.4474628881673]
  Diffusion models (DMs) represent state-of-the-art generative models for continuous inputs. We introduce a novel generative modeling framework grounded in textbfphase space dynamics Our framework demonstrates the capability to generate realistic data points at an early stage of dynamics propagation.
  arXiv  Detail & Related papers  (2023-10-11T18:38:28Z)
• A generic diffusion-based approach for 3D human pose prediction in the wild [68.00961210467479]
  3D human pose forecasting, i.e., predicting a sequence of future human 3D poses given a sequence of past observed ones, is a challenging-temporal task. We provide a unified formulation in which incomplete elements (no matter in the prediction or observation) are treated as noise and propose a conditional diffusion model that denoises them and forecasts plausible poses. We investigate our findings on four standard datasets and obtain significant improvements over the state-of-the-art.
  arXiv  Detail & Related papers  (2022-10-11T17:59:54Z)
• Multivariate Time Series Forecasting with Dynamic Graph Neural ODEs [65.18780403244178]
  We propose a continuous model to forecast Multivariate Time series with dynamic Graph neural Ordinary Differential Equations (MTGODE) Specifically, we first abstract multivariate time series into dynamic graphs with time-evolving node features and unknown graph structures. Then, we design and solve a neural ODE to complement missing graph topologies and unify both spatial and temporal message passing.
  arXiv  Detail & Related papers  (2022-02-17T02:17:31Z)
• A Spatio-temporal Transformer for 3D Human Motion Prediction [39.31212055504893]
  We propose a Transformer-based architecture for the task of generative modelling of 3D human motion. We empirically show that this effectively learns the underlying motion dynamics and reduces error accumulation over time observed in auto-gressive models.
  arXiv  Detail & Related papers  (2020-04-18T19:49:28Z)
• Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition [79.33539539956186]
  We propose a simple method to disentangle multi-scale graph convolutions and a unified spatial-temporal graph convolutional operator named G3D. By coupling these proposals, we develop a powerful feature extractor named MS-G3D based on which our model outperforms previous state-of-the-art methods on three large-scale datasets.
  arXiv  Detail & Related papers  (2020-03-31T11:28:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.