Split&Splat: Zero-Shot Panoptic Segmentation via Explicit Instance Modeling and 3D Gaussian Splatting

• URL: http://arxiv.org/abs/2602.03809v1
• Date: Sun, 01 Feb 2026 20:10:37 GMT
• Title: Split&Splat: Zero-Shot Panoptic Segmentation via Explicit Instance Modeling and 3D Gaussian Splatting
• Authors: Leonardo Monchieri, Elena Camuffo, Francesco Barbato, Pietro Zanuttigh, Simone Milani,
• Abstract summary: Split&Splat is a framework for panoptic scene reconstruction using 3DGS.<n>Split&Splat tackles the problem by first segmenting the scene and then reconstructing each object individually.<n>This design naturally supports downstream tasks and allows Split&Splat to achieve state-of-the-art performance on the ScanNetv2 segmentation benchmark.
• Score: 18.506942200662575
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: 3D Gaussian Splatting (GS) enables fast and high-quality scene reconstruction, but it lacks an object-consistent and semantically aware structure. We propose Split&Splat, a framework for panoptic scene reconstruction using 3DGS. Our approach explicitly models object instances. It first propagates instance masks across views using depth, thus producing view-consistent 2D masks. Each object is then reconstructed independently and merged back into the scene while refining its boundaries. Finally, instance-level semantic descriptors are embedded in the reconstructed objects, supporting various applications, including panoptic segmentation, object retrieval, and 3D editing. Unlike existing methods, Split&Splat tackles the problem by first segmenting the scene and then reconstructing each object individually. This design naturally supports downstream tasks and allows Split&Splat to achieve state-of-the-art performance on the ScanNetv2 segmentation benchmark.

Related papers

• Seeing Through Clutter: Structured 3D Scene Reconstruction via Iterative Object Removal [11.166147692815931]
  We present SeeingThroughClutter, a method for reconstructing structured 3D representations from single images by segmenting and modeling objects individually.<n>We address this by introducing an iterative object removal and reconstruction pipeline that decomposes complex scenes into a sequence of simpler subtasks.<n>Our method requires no task-specific training and benefits directly from ongoing advances in foundation models.
  arXiv  Detail & Related papers  (2026-02-03T22:37:43Z)
• GeoSAM2: Unleashing the Power of SAM2 for 3D Part Segmentation [81.0871900167463]
  We introduce GeoSAM2, a prompt-controllable framework for 3D part segmentation.<n>Given a textureless object, we render normal and point maps from predefined viewpoints.<n>We accept simple 2D prompts - clicks or boxes - to guide part selection.<n>The predicted masks are back-projected to the object and aggregated across views.
  arXiv  Detail & Related papers  (2025-08-19T17:58:51Z)
• ObjectGS: Object-aware Scene Reconstruction and Scene Understanding via Gaussian Splatting [54.92763171355442]
  ObjectGS is an object-aware framework that unifies 3D scene reconstruction with semantic understanding.<n>We show through experiments that ObjectGS outperforms state-of-the-art methods on open-vocabulary and panoptic segmentation tasks.
  arXiv  Detail & Related papers  (2025-07-21T10:06:23Z)
• Segment then Splat: Unified 3D Open-Vocabulary Segmentation via Gaussian Splatting [15.849458071604758]
  Open-vocabulary querying in 3D space is crucial for enabling more intelligent perception in applications such as robotics, autonomous systems, and augmented reality.<n>Most existing methods rely on 2D pixel-level parsing, leading to multi-view inconsistencies and poor 3D object retrieval.<n>We propose Segment then, a 3D-aware open vocabulary segmentation approach for both static and dynamic scenes based on Gaussian Splatting.
  arXiv  Detail & Related papers  (2025-03-28T07:36:51Z)
• 3D Part Segmentation via Geometric Aggregation of 2D Visual Features [57.20161517451834]
  Supervised 3D part segmentation models are tailored for a fixed set of objects and parts, limiting their transferability to open-set, real-world scenarios.<n>Recent works have explored vision-language models (VLMs) as a promising alternative, using multi-view rendering and textual prompting to identify object parts.<n>To address these limitations, we propose COPS, a COmprehensive model for Parts that blends semantics extracted from visual concepts and 3D geometry to effectively identify object parts.
  arXiv  Detail & Related papers  (2024-12-05T15:27:58Z)
• 3D-GRES: Generalized 3D Referring Expression Segmentation [77.10044505645064]
  3D Referring Expression (3D-RES) is dedicated to segmenting a specific instance within a 3D space based on a natural language description. Generalized 3D Referring Expression (3D-GRES) extends the capability to segment any number of instances based on natural language instructions.
  arXiv  Detail & Related papers  (2024-07-30T08:59:05Z)
• Part2Object: Hierarchical Unsupervised 3D Instance Segmentation [31.44173252707684]
  Unsupervised 3D instance segmentation aims to segment objects from a 3D point cloud without any annotations. Part2Object employs multi-layer clustering from points to object parts and objects, allowing objects to manifest at any layer. We propose Hi-Mask3D to support hierarchical 3D object part and instance segmentation.
  arXiv  Detail & Related papers  (2024-07-14T05:18:15Z)
• SAI3D: Segment Any Instance in 3D Scenes [68.57002591841034]
  We introduce SAI3D, a novel zero-shot 3D instance segmentation approach. Our method partitions a 3D scene into geometric primitives, which are then progressively merged into 3D instance segmentations. Empirical evaluations on ScanNet, Matterport3D and the more challenging ScanNet++ datasets demonstrate the superiority of our approach.
  arXiv  Detail & Related papers  (2023-12-17T09:05:47Z)
• Iterative Superquadric Recomposition of 3D Objects from Multiple Views [77.53142165205283]
  We propose a framework, ISCO, to recompose an object using 3D superquadrics as semantic parts directly from 2D views. Our framework iteratively adds new superquadrics wherever the reconstruction error is high. It provides consistently more accurate 3D reconstructions, even from images in the wild.
  arXiv  Detail & Related papers  (2023-09-05T10:21:37Z)
• A One Stop 3D Target Reconstruction and multilevel Segmentation Method [0.0]
  We propose an open-source one stop 3D target reconstruction and multilevel segmentation framework (OSTRA) OSTRA performs segmentation on 2D images, tracks multiple instances with segmentation labels in the image sequence, and then reconstructs labelled 3D objects or multiple parts with Multi-View Stereo (MVS) or RGBD-based 3D reconstruction methods. Our method opens up a new avenue for reconstructing 3D targets embedded with rich multi-scale segmentation information in complex scenes.
  arXiv  Detail & Related papers  (2023-08-14T07:12:31Z)
• AutoSweep: Recovering 3D Editable Objectsfrom a Single Photograph [54.701098964773756]
  We aim to recover 3D objects with semantic parts and can be directly edited. Our work makes an attempt towards recovering two types of primitive-shaped objects, namely, generalized cuboids and generalized cylinders. Our algorithm can recover high quality 3D models and outperforms existing methods in both instance segmentation and 3D reconstruction.
  arXiv  Detail & Related papers  (2020-05-27T12:16:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.