HoloPart: Generative 3D Part Amodal Segmentation

• URL: http://arxiv.org/abs/2504.07943v1
• Date: Thu, 10 Apr 2025 17:53:31 GMT
• Title: HoloPart: Generative 3D Part Amodal Segmentation
• Authors: Yunhan Yang, Yuan-Chen Guo, Yukun Huang, Zi-Xin Zou, Zhipeng Yu, Yangguang Li, Yan-Pei Cao, Xihui Liu,
• Abstract summary: 3D part amodal segmentation is a challenging but crucial task for 3D content creation and understanding.<n>Inspired by 2D amodal segmentation, we introduce this novel task to the 3D domain and propose a practical, two-stage approach.<n>We introduce new benchmarks based on the ABO and Part-Tiny datasets and demonstrate that HoloPart significantly outperforms state-of-the-art shape completion methods.
• Score: 23.84639726216676
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: 3D part amodal segmentation--decomposing a 3D shape into complete, semantically meaningful parts, even when occluded--is a challenging but crucial task for 3D content creation and understanding. Existing 3D part segmentation methods only identify visible surface patches, limiting their utility. Inspired by 2D amodal segmentation, we introduce this novel task to the 3D domain and propose a practical, two-stage approach, addressing the key challenges of inferring occluded 3D geometry, maintaining global shape consistency, and handling diverse shapes with limited training data. First, we leverage existing 3D part segmentation to obtain initial, incomplete part segments. Second, we introduce HoloPart, a novel diffusion-based model, to complete these segments into full 3D parts. HoloPart utilizes a specialized architecture with local attention to capture fine-grained part geometry and global shape context attention to ensure overall shape consistency. We introduce new benchmarks based on the ABO and PartObjaverse-Tiny datasets and demonstrate that HoloPart significantly outperforms state-of-the-art shape completion methods. By incorporating HoloPart with existing segmentation techniques, we achieve promising results on 3D part amodal segmentation, opening new avenues for applications in geometry editing, animation, and material assignment.

Related papers

• PARTFIELD: Learning 3D Feature Fields for Part Segmentation and Beyond [70.95930509071451]
  PartField is a feedforward approach for learning part-based 3D features. PartField is up to 20% more accurate and often orders of magnitude faster than other recent class-agnostic part-segmentation methods.
  arXiv  Detail & Related papers  (2025-04-15T17:58:16Z)
• Open-Vocabulary Semantic Part Segmentation of 3D Human [4.380538063753977]
  We present the first open-vocabulary segmentation method capable of handling 3D human. Our framework can segment the human category into desired fine-grained parts based on the textual prompt. Our method can be directly applied to various 3D representations including meshes, point clouds, and 3D Gaussian Splatting.
  arXiv  Detail & Related papers  (2025-02-27T05:47:05Z)
• PartGen: Part-level 3D Generation and Reconstruction with Multi-View Diffusion Models [63.1432721793683]
  We introduce PartGen, a novel approach that generates 3D objects composed of meaningful parts starting from text, an image, or an unstructured 3D object.<n>We evaluate our method on generated and real 3D assets and show that it outperforms segmentation and part-extraction baselines by a large margin.
  arXiv  Detail & Related papers  (2024-12-24T18:59:43Z)
• 3D Part Segmentation via Geometric Aggregation of 2D Visual Features [57.20161517451834]
  Supervised 3D part segmentation models are tailored for a fixed set of objects and parts, limiting their transferability to open-set, real-world scenarios.<n>Recent works have explored vision-language models (VLMs) as a promising alternative, using multi-view rendering and textual prompting to identify object parts.<n>To address these limitations, we propose COPS, a COmprehensive model for Parts that blends semantics extracted from visual concepts and 3D geometry to effectively identify object parts.
  arXiv  Detail & Related papers  (2024-12-05T15:27:58Z)
• SAMPart3D: Segment Any Part in 3D Objects [23.97392239910013]
  3D part segmentation is a crucial and challenging task in 3D perception, playing a vital role in applications such as robotics, 3D generation, and 3D editing. Recent methods harness the powerful Vision Language Models (VLMs) for 2D-to-3D knowledge distillation, achieving zero-shot 3D part segmentation. In this work, we introduce SAMPart3D, a scalable zero-shot 3D part segmentation framework that segments any 3D object into semantic parts at multiple granularities.
  arXiv  Detail & Related papers  (2024-11-11T17:59:10Z)
• MeshSegmenter: Zero-Shot Mesh Semantic Segmentation via Texture Synthesis [27.703204488877038]
  MeshSegmenter is a framework designed for zero-shot 3D semantic segmentation. It delivers accurate 3D segmentation across diverse meshes and segment descriptions.
  arXiv  Detail & Related papers  (2024-07-18T16:50:59Z)
• 3x2: 3D Object Part Segmentation by 2D Semantic Correspondences [33.99493183183571]
  We propose to leverage a few annotated 3D shapes or richly annotated 2D datasets to perform 3D object part segmentation. We present our novel approach, termed 3-By-2 that achieves SOTA performance on different benchmarks with various granularity levels.
  arXiv  Detail & Related papers  (2024-07-12T19:08:00Z)
• Reasoning3D -- Grounding and Reasoning in 3D: Fine-Grained Zero-Shot Open-Vocabulary 3D Reasoning Part Segmentation via Large Vision-Language Models [20.277479473218513]
  We introduce a new task: Zero-Shot 3D Reasoning for parts searching and localization for objects. We design a simple baseline method, Reasoning3D, with the capability to understand and execute complex commands. We show that Reasoning3D can effectively localize and highlight parts of 3D objects based on implicit textual queries.
  arXiv  Detail & Related papers  (2024-05-29T17:56:07Z)
• Segment Any 3D Gaussians [85.93694310363325]
  This paper presents SAGA, a highly efficient 3D promptable segmentation method based on 3D Gaussian Splatting (3D-GS)<n>Given 2D visual prompts as input, SAGA can segment the corresponding 3D target represented by 3D Gaussians within 4 ms.<n>We show that SAGA achieves real-time multi-granularity segmentation with quality comparable to state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-12-01T17:15:24Z)
• PartSLIP: Low-Shot Part Segmentation for 3D Point Clouds via Pretrained Image-Language Models [56.324516906160234]
  Generalizable 3D part segmentation is important but challenging in vision and robotics. This paper explores an alternative way for low-shot part segmentation of 3D point clouds by leveraging a pretrained image-language model, GLIP. We transfer the rich knowledge from 2D to 3D through GLIP-based part detection on point cloud rendering and a novel 2D-to-3D label lifting algorithm.
  arXiv  Detail & Related papers  (2022-12-03T06:59:01Z)
• Discovering 3D Parts from Image Collections [98.16987919686709]
  We tackle the problem of 3D part discovery from only 2D image collections. Instead of relying on manually annotated parts for supervision, we propose a self-supervised approach. Our key insight is to learn a novel part shape prior that allows each part to fit an object shape faithfully while constrained to have simple geometry.
  arXiv  Detail & Related papers  (2021-07-28T20:29:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.