SceneTok: A Compressed, Diffusable Token Space for 3D Scenes

• URL: http://arxiv.org/abs/2602.18882v2
• Date: Thu, 26 Feb 2026 20:54:19 GMT
• Title: SceneTok: A Compressed, Diffusable Token Space for 3D Scenes
• Abstract summary: We present SceneTok, a novel tokenizer for encoding view sets of scenes into a compressed and diffusable set of unstructured tokens.<n>We show that the compression is 1-3 orders of magnitude stronger than for other representations while still reaching state-of-the-art reconstruction quality.
• Score: 18.094424146923892
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present SceneTok, a novel tokenizer for encoding view sets of scenes into a compressed and diffusable set of unstructured tokens. Existing approaches for 3D scene representation and generation commonly use 3D data structures or view-aligned fields. In contrast, we introduce the first method that encodes scene information into a small set of permutation-invariant tokens that is disentangled from the spatial grid. The scene tokens are predicted by a multi-view tokenizer given many context views and rendered into novel views by employing a light-weight rectified flow decoder. We show that the compression is 1-3 orders of magnitude stronger than for other representations while still reaching state-of-the-art reconstruction quality. Further, our representation can be rendered from novel trajectories, including ones deviating from the input trajectory, and we show that the decoder gracefully handles uncertainty. Finally, the highly-compressed set of unstructured latent scene tokens enables simple and efficient scene generation in 5 seconds, achieving a much better quality-speed trade-off than previous paradigms.

Related papers

• CoVeR: Coverage-Based Token Pruning for Multi-View 3D Reasoning in VLMs [58.22241532542095]
  Representing a 3D scene as multi-view images allows 2D VLMs to reason in 3D by reusing priors from pre-training, sidestepping the scarcity of annotated 3D data.<n>Existing visual token pruners fall into two families, each limited in the 3D multi-view setting.<n>We show that spatial coverage is associated with 3D reasoning performance and introduce CoVeR, a deterministic, training-free selector that uses only token coordinates, with no learned signals.
  arXiv  Detail & Related papers  (2026-09-08T07:16:37Z)
• ZipTok3D: High-Fidelity 3D Tokenization with Compact Token Prefixes [38.735159381505014]
  ZipTok3D is a 3D tokenizer designed for high-fidelity reconstruction from extremely short token sequences.<n>With the same token dimension, ZipTok3D achieves reconstruction quality comparable to the 32-token COD-VAE baseline.
  arXiv  Detail & Related papers  (2026-09-01T18:07:49Z)
• LumiTokens: 3D Relighting via Token-Space Lighting Transformation [19.630351342370354]
  We present LumiTokens, a framework that formulates 3D relighting as a direct transformation on latent scene tokens.<n>Our model introduces a Scene Token Editor that processes scene tokens jointly with light-ray tokens through self-attention.<n>Experiments demonstrate that LumiTokens achieves comparable or superior relighting quality to other methods.
  arXiv  Detail & Related papers  (2026-08-18T18:00:21Z)
• DriveTok: 3D Driving Scene Tokenization for Unified Multi-View Reconstruction and Understanding [75.41918200553525]
  DriveTok is an efficient 3D driving scene tokenizer for unified multi-view reconstruction and understanding.<n>For decoding, we employ a multi-view transformer to reconstruct multi-view features from the scene tokens.<n>We also add a 3D head directly on the scene tokens for 3D semantic occupancy prediction.
  arXiv  Detail & Related papers  (2026-03-19T17:58:22Z)
• CLiFT: Compressive Light-Field Tokens for Compute-Efficient and Adaptive Neural Rendering [20.916387646203273]
  This paper proposes a neural rendering approach that represents a scene as "compressed light-field tokens (CLiFTs)"<n>CLiFT enables compute-efficient rendering by compressed tokens, while being capable of changing the number of tokens to represent a scene or render a novel view with one trained network.
  arXiv  Detail & Related papers  (2025-07-11T17:38:52Z)
• Images are Worth Variable Length of Representations [13.136831256070343]
  Most vision encoders map images into a fixed-length sequence of tokens, overlooking the fact that different images contain varying amounts of information.<n>We propose DOVE, a dynamic vision encoder that produces a variable number of visual tokens to reconstruct each image.<n>Our results show that DOVE significantly reduces the average number of tokens while maintaining high reconstruction quality.
  arXiv  Detail & Related papers  (2025-06-04T07:40:33Z)
• Self-Supervised and Generalizable Tokenization for CLIP-Based 3D Understanding [87.68271178167373]
  We present a universal 3D tokenizer designed for scale-invariant representation learning with a frozen CLIP backbone.<n>S4Token is a tokenization pipeline that produces semantically-informed tokens regardless of scene scale.
  arXiv  Detail & Related papers  (2025-05-24T18:26:30Z)
• BloomScene: Lightweight Structured 3D Gaussian Splatting for Crossmodal Scene Generation [54.12899218104669]
  3D scenes have highly complex structures and need to ensure that the output is dense, coherent, and contains all necessary structures.<n>Current 3D scene generation methods rely on pre-trained text-to-image diffusion models and monocular depth estimators.<n>We propose BloomScene, a lightweight structured 3D Gaussian splatting for crossmodal scene generation.
  arXiv  Detail & Related papers  (2025-01-15T11:33:34Z)
• SCube: Instant Large-Scale Scene Reconstruction using VoxSplats [55.383993296042526]
  We present SCube, a novel method for reconstructing large-scale 3D scenes (geometry, appearance, and semantics) from a sparse set of posed images. Our method encodes reconstructed scenes using a novel representation VoxSplat, which is a set of 3D Gaussians supported on a high-resolution sparse-voxel scaffold.
  arXiv  Detail & Related papers  (2024-10-26T00:52:46Z)
• ElasticTok: Adaptive Tokenization for Image and Video [109.75935878130582]
  We introduce ElasticTok, a method that conditions on prior frames to adaptively encode a frame into a variable number of tokens.<n>During inference, ElasticTok can dynamically allocate tokens when needed.<n>Our evaluations on images and video demonstrate the effectiveness of our approach in efficient token usage.
  arXiv  Detail & Related papers  (2024-10-10T20:54:15Z)
• One-Shot Neural Fields for 3D Object Understanding [112.32255680399399]
  We present a unified and compact scene representation for robotics. Each object in the scene is depicted by a latent code capturing geometry and appearance. This representation can be decoded for various tasks such as novel view rendering, 3D reconstruction, and stable grasp prediction.
  arXiv  Detail & Related papers  (2022-10-21T17:33:14Z)
• PINs: Progressive Implicit Networks for Multi-Scale Neural Representations [68.73195473089324]
  We propose a progressive positional encoding, exposing a hierarchical structure to incremental sets of frequency encodings. Our model accurately reconstructs scenes with wide frequency bands and learns a scene representation at progressive level of detail. Experiments on several 2D and 3D datasets show improvements in reconstruction accuracy, representational capacity and training speed compared to baselines.
  arXiv  Detail & Related papers  (2022-02-09T20:33:37Z)
• Scene Representation Transformer: Geometry-Free Novel View Synthesis Through Set-Latent Scene Representations [48.05445941939446]
  A classical problem in computer vision is to infer a 3D scene representation from few images that can be used to render novel views at interactive rates. We propose the Scene Representation Transformer (SRT), a method which processes posed or unposed RGB images of a new area. We show that this method outperforms recent baselines in terms of PSNR and speed on synthetic datasets.
  arXiv  Detail & Related papers  (2021-11-25T16:18:56Z)
• 3D Scene Compression through Entropy Penalized Neural Representation Functions [19.277502420759653]
  novel visual media enable the viewer to explore a 3D scene from arbitrary viewpoints, by interpolating between a discrete set of original views. These types of applications require much larger amounts of storage space, which we seek to reduce. Existing approaches for compressing 3D scenes are based on a separation of compression and rendering. We unify these steps by directly compressing an implicit representation of the scene, a function that maps spatial coordinates to a radiance vector field, which can then be queried to render arbitrary viewpoints. Our method significantly outperforms a state-of-the-art conventional approach for scene compression, achieving simultaneously higher quality reconstruction
  arXiv  Detail & Related papers  (2021-04-26T10:36:47Z)
• Semantic Implicit Neural Scene Representations With Semi-Supervised Training [47.61092265963234]
  We show that implicit neural scene representations can be leveraged to perform per-point semantic segmentation. Our method is simple, general, and only requires a few tens of labeled 2D segmentation masks. We explore two novel applications for this semantically aware implicit neural scene representation.
  arXiv  Detail & Related papers  (2020-03-28T00:43:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.