iSeg3D: An Interactive 3D Shape Segmentation Tool

• URL: http://arxiv.org/abs/2112.12988v1
• Date: Fri, 24 Dec 2021 08:15:52 GMT
• Title: iSeg3D: An Interactive 3D Shape Segmentation Tool
• Authors: Sucheng Qian, Liu Liu, Wenqiang Xu, Cewu Lu
• Abstract summary: We propose an effective annotation tool, named iSeg for 3D shape. Under our observation, most objects can be considered as the composition of finite primitive shapes. We train iSeg3D model on our built primitive-composed shape data to learn the geometric prior knowledge in a self-supervised manner.
• Score: 48.784624011210475
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A large-scale dataset is essential for learning good features in 3D shape understanding, but there are only a few datasets that can satisfy deep learning training. One of the major reasons is that current tools for annotating per-point semantic labels using polygons or scribbles are tedious and inefficient. To facilitate segmentation annotations in 3D shapes, we propose an effective annotation tool, named iSeg for 3D shape. It can obtain a satisfied segmentation result with minimal human clicks (< 10). Under our observation, most objects can be considered as the composition of finite primitive shapes, and we train iSeg3D model on our built primitive-composed shape data to learn the geometric prior knowledge in a self-supervised manner. Given human interactions, the learned knowledge can be used to segment parts on arbitrary shapes, in which positive clicks help associate the primitives into the semantic parts and negative clicks can avoid over-segmentation. Besides, We also provide an online human-in-loop fine-tuning module that enables the model perform better segmentation with less clicks. Experiments demonstrate the effectiveness of iSeg3D on PartNet shape segmentation. Data and codes will be made publicly available.

Related papers

• PARIS3D: Reasoning-based 3D Part Segmentation Using Large Multimodal Model [19.333506797686695]
  We introduce a novel segmentation task known as reasoning part segmentation for 3D objects. We output a segmentation mask based on complex and implicit textual queries about specific parts of a 3D object. We propose a model that is capable of segmenting parts of 3D objects based on implicit textual queries and generating natural language explanations.
  arXiv  Detail & Related papers  (2024-04-04T23:38:45Z)
• Semi-Weakly Supervised Object Kinematic Motion Prediction [56.282759127180306]
  Given a 3D object, kinematic motion prediction aims to identify the mobile parts as well as the corresponding motion parameters. We propose a graph neural network to learn the map between hierarchical part-level segmentation and mobile parts parameters. The network predictions yield a large scale of 3D objects with pseudo labeled mobility information.
  arXiv  Detail & Related papers  (2023-03-31T02:37:36Z)
• PartSLIP: Low-Shot Part Segmentation for 3D Point Clouds via Pretrained Image-Language Models [56.324516906160234]
  Generalizable 3D part segmentation is important but challenging in vision and robotics. This paper explores an alternative way for low-shot part segmentation of 3D point clouds by leveraging a pretrained image-language model, GLIP. We transfer the rich knowledge from 2D to 3D through GLIP-based part detection on point cloud rendering and a novel 2D-to-3D label lifting algorithm.
  arXiv  Detail & Related papers  (2022-12-03T06:59:01Z)
• MvDeCor: Multi-view Dense Correspondence Learning for Fine-grained 3D Segmentation [91.6658845016214]
  We propose to utilize self-supervised techniques in the 2D domain for fine-grained 3D shape segmentation tasks. We render a 3D shape from multiple views, and set up a dense correspondence learning task within the contrastive learning framework. As a result, the learned 2D representations are view-invariant and geometrically consistent.
  arXiv  Detail & Related papers  (2022-08-18T00:48:15Z)
• Semi-supervised 3D shape segmentation with multilevel consistency and part substitution [21.075426681857024]
  We propose an effective semi-supervised method for learning 3D segmentations from a few labeled 3D shapes and a large amount of unlabeled 3D data. For the unlabeled data, we present a novel multilevel consistency loss to enforce consistency of network predictions between perturbed copies of a 3D shape. For the labeled data, we develop a simple yet effective part substitution scheme to augment the labeled 3D shapes with more structural variations to enhance training.
  arXiv  Detail & Related papers  (2022-04-19T11:48:24Z)
• SurFit: Learning to Fit Surfaces Improves Few Shot Learning on Point Clouds [48.61222927399794]
  SurFit is a simple approach for label efficient learning of 3D shape segmentation networks. It is based on a self-supervised task of decomposing the surface of a 3D shape into geometric primitives.
  arXiv  Detail & Related papers  (2021-12-27T23:55:36Z)
• Meta-Learning 3D Shape Segmentation Functions [16.119694625781992]
  We introduce an auxiliary deep neural network as a meta-learner which takes as input a 3D shape and predicts the prior over the respective 3D segmentation function space. We show in experiments that our meta-learning approach, denoted as Meta-3DSeg, leads to improvements on unsupervised 3D shape segmentation.
  arXiv  Detail & Related papers  (2021-10-08T01:50:54Z)
• 3D Meta-Segmentation Neural Network [12.048487830494107]
  We present a novel meta-learning strategy that regards the 3D shape segmentation function as a task. By training over a number of 3D part segmentation tasks, our method is capable to learn the prior over the respective 3D segmentation function space. We demonstrate that our model achieves superior part segmentation performance with the few-shot setting on the widely used dataset: ShapeNet.
  arXiv  Detail & Related papers  (2021-10-08T01:47:54Z)
• Discovering 3D Parts from Image Collections [98.16987919686709]
  We tackle the problem of 3D part discovery from only 2D image collections. Instead of relying on manually annotated parts for supervision, we propose a self-supervised approach. Our key insight is to learn a novel part shape prior that allows each part to fit an object shape faithfully while constrained to have simple geometry.
  arXiv  Detail & Related papers  (2021-07-28T20:29:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.