Scenes as Objects, Not Primitives: Instance-Structured 3D Tokenization from Unposed Views

• URL: http://arxiv.org/abs/2606.29513v1
• Date: Sun, 28 Jun 2026 17:21:45 GMT
• Title: Scenes as Objects, Not Primitives: Instance-Structured 3D Tokenization from Unposed Views
• Abstract summary: A 3D scene is understood through its objects, not the primitives that compose them.<n>We propose a framework that decomposes a scene into instance-structured 3D token groups directly from unposed multi-view images.
• Score: 35.7336094120467
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A 3D scene is understood through its objects, not the primitives that compose them. Yet feed-forward reconstruction methods output dense, unstructured sets of points or Gaussians, leaving object-level structure to be recovered after the fact. We propose a feed-forward framework that decomposes a scene into instance-structured 3D token groups directly from unposed multi-view images -- compact object-centric units from which reconstruction, segmentation, and manipulation all follow. Each token group pairs an instance token capturing entity-level identity with anchor tokens that encode local geometry and appearance, which are decoded into a set of 3D Gaussians. This two-level factorization decouples object identity from local appearance, making object instances a native interface of the representation rather than a derived product. The token groups are learned through differentiable rendering with joint reconstruction and segmentation supervision, requiring no 3D annotations. Our feed-forward model surpasses per-scene optimization baselines in class-agnostic instance segmentation while remaining competitive in novel view synthesis. Beyond these metrics, the same token groups directly unlock instance-level scene editing -- removing, translating, or inserting objects by operating on their groups -- as well as efficient open-vocabulary 3D instance retrieval, where retrieval complexity scales with the number of instances rather than primitives.

Related papers

• GroupForward: Building Referable 3D Scenes via Instance-Grouped Feed-Forward Gaussian Splatting [46.95070188813926]
  GroupForward reconstructs geometry, appearance, instance structure, and semantics from sparse, unposed, and uncalibrated images.<n> RSRF constructs an instance-grouped 3D scene graph and retrieves candidate instances for a given referring expression.<n> Experiments on semantic reconstruction and referential reasoning demonstrate the effectiveness of our instance-grouped reconstruction and reasoning framework.
  arXiv  Detail & Related papers  (2026-08-18T08:55:52Z)
• InstanceSplat: Instance-Aware Feed-Forward 3D Gaussian Splatting for Scene Understanding [7.875166305092363]
  We present InstanceSplat, a unified feed-forward 3DGS framework for generalizable 3D reconstruction and instance-aware scene understanding.<n>In a single forward pass, InstanceSplat constructs an instance-aware Gaussian representation that jointly encodes appearance, geometry, instance identity, and language-aligned semantics.
  arXiv  Detail & Related papers  (2026-08-07T12:02:48Z)
• OCH3R: Object-Centric Holistic 3D Reconstruction [36.45885767128656]
  We introduce OCH3R, a unified framework for Object-Centric Holistic 3D Reconstruction from a single RGB image.<n>OCH3R performs one forward pass to simultaneously predict all object instances with their 6D poses and detailed 3D reconstructions.<n>On standard indoor benchmarks, OCH3R achieves state-of-the-art performance across monocular depth estimation, open-vocabulary semantic segmentation, and RGB-only category-level 6D pose estimation.
  arXiv  Detail & Related papers  (2026-05-13T05:17:51Z)
• FAST3DIS: Feed-forward Anchored Scene Transformer for 3D Instance Segmentation [15.271467111162714]
  FAST3DIS is an end-to-end approach that effectively bypasses post-hoc clustering.<n>We introduce a 3D-anchored, query-based Transformer architecture built upon a foundational depth backbone.<n> Experiments on complex indoor 3D datasets demonstrate that our approach achieves competitive segmentation accuracy.
  arXiv  Detail & Related papers  (2026-03-27T00:45:31Z)
• Group3D: MLLM-Driven Semantic Grouping for Open-Vocabulary 3D Object Detection [23.34626996348395]
  Group3D is a multi-view open-vocabulary 3D detection framework.<n>It integrates semantic constraints directly into the instance construction process.<n>Group3D achieves state-of-the-art performance in multi-view open-vocabulary 3D detection.
  arXiv  Detail & Related papers  (2026-03-23T13:01:14Z)
• Seeing Through Clutter: Structured 3D Scene Reconstruction via Iterative Object Removal [11.166147692815931]
  We present SeeingThroughClutter, a method for reconstructing structured 3D representations from single images by segmenting and modeling objects individually.<n>We address this by introducing an iterative object removal and reconstruction pipeline that decomposes complex scenes into a sequence of simpler subtasks.<n>Our method requires no task-specific training and benefits directly from ongoing advances in foundation models.
  arXiv  Detail & Related papers  (2026-02-03T22:37:43Z)
• IGGT: Instance-Grounded Geometry Transformer for Semantic 3D Reconstruction [82.53307702809606]
  Humans naturally perceive the geometric structure and semantic content of a 3D world as intertwined dimensions.<n>We propose InstanceGrounded Geometry Transformer (IGGT) to unify the knowledge for both spatial reconstruction and instance-level contextual understanding.
  arXiv  Detail & Related papers  (2025-10-26T14:57:44Z)
• ObjectGS: Object-aware Scene Reconstruction and Scene Understanding via Gaussian Splatting [54.92763171355442]
  ObjectGS is an object-aware framework that unifies 3D scene reconstruction with semantic understanding.<n>We show through experiments that ObjectGS outperforms state-of-the-art methods on open-vocabulary and panoptic segmentation tasks.
  arXiv  Detail & Related papers  (2025-07-21T10:06:23Z)
• Object-X: Learning to Reconstruct Multi-Modal 3D Object Representations [114.57192386025373]
  Object-X is a versatile multi-modal 3D representation framework.<n>It can encoding rich object embeddings and decoding them back into geometric and visual reconstructions.<n>It supports a range of downstream tasks, including scene alignment, single-image 3D object reconstruction, and localization.
  arXiv  Detail & Related papers  (2025-06-05T09:14:42Z)
• Iterative Superquadric Recomposition of 3D Objects from Multiple Views [77.53142165205283]
  We propose a framework, ISCO, to recompose an object using 3D superquadrics as semantic parts directly from 2D views. Our framework iteratively adds new superquadrics wherever the reconstruction error is high. It provides consistently more accurate 3D reconstructions, even from images in the wild.
  arXiv  Detail & Related papers  (2023-09-05T10:21:37Z)
• Shelf-Supervised Mesh Prediction in the Wild [54.01373263260449]
  We propose a learning-based approach to infer 3D shape and pose of object from a single image. We first infer a volumetric representation in a canonical frame, along with the camera pose. The coarse volumetric prediction is then converted to a mesh-based representation, which is further refined in the predicted camera frame.
  arXiv  Detail & Related papers  (2021-02-11T18:57:10Z)
• Convolutional Occupancy Networks [88.48287716452002]
  We propose Convolutional Occupancy Networks, a more flexible implicit representation for detailed reconstruction of objects and 3D scenes. By combining convolutional encoders with implicit occupancy decoders, our model incorporates inductive biases, enabling structured reasoning in 3D space. We empirically find that our method enables the fine-grained implicit 3D reconstruction of single objects, scales to large indoor scenes, and generalizes well from synthetic to real data.
  arXiv  Detail & Related papers  (2020-03-10T10:17:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.