Seg&Struct: The Interplay Between Part Segmentation and Structure Inference for 3D Shape Parsing

• URL: http://arxiv.org/abs/2211.00382v1
• Date: Tue, 1 Nov 2022 10:59:15 GMT
• Title: Seg&Struct: The Interplay Between Part Segmentation and Structure Inference for 3D Shape Parsing
• Authors: Jeonghyun Kim, Kaichun Mo, Minhyuk Sung, Woontack Woo
• Abstract summary: Seg&Struct is a supervised learning framework leveraging the interplay between part segmentation and structure inference. We present how these two tasks can be best combined while fully utilizing supervision to improve performance.
• Score: 23.8184215719129
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose Seg&Struct, a supervised learning framework leveraging the interplay between part segmentation and structure inference and demonstrating their synergy in an integrated framework. Both part segmentation and structure inference have been extensively studied in the recent deep learning literature, while the supervisions used for each task have not been fully exploited to assist the other task. Namely, structure inference has been typically conducted with an autoencoder that does not leverage the point-to-part associations. Also, segmentation has been mostly performed without structural priors that tell the plausibility of the output segments. We present how these two tasks can be best combined while fully utilizing supervision to improve performance. Our framework first decomposes a raw input shape into part segments using an off-the-shelf algorithm, whose outputs are then mapped to nodes in a part hierarchy, establishing point-to-part associations. Following this, ours predicts the structural information, e.g., part bounding boxes and part relationships. Lastly, the segmentation is rectified by examining the confusion of part boundaries using the structure-based part features. Our experimental results based on the StructureNet and PartNet demonstrate that the interplay between the two tasks results in remarkable improvements in both tasks: 27.91% in structure inference and 0.5% in segmentation.

Related papers

• Fine-Grained Image-Text Correspondence with Cost Aggregation for Open-Vocabulary Part Segmentation [24.071471822239854]
  Open-Vocabulary Part (OVPS) is an emerging field for recognizing fine-grained parts in unseen categories. We identify two primary challenges in OVPS: the difficulty in aligning part-level image-text correspondence, and the lack of structural understanding in segmenting object parts. We propose PartCATSeg, a novel framework that integrates object-aware part-level cost aggregation, compositional loss, and structural guidance from DINO.
  arXiv  Detail & Related papers  (2025-01-16T17:40:19Z)
• From Text Segmentation to Smart Chaptering: A Novel Benchmark for Structuring Video Transcriptions [63.11097464396147]
  We introduce a novel benchmark YTSeg focusing on spoken content that is inherently more unstructured and both topically and structurally diverse. We also introduce an efficient hierarchical segmentation model MiniSeg, that outperforms state-of-the-art baselines.
  arXiv  Detail & Related papers  (2024-02-27T15:59:37Z)
• Topic Segmentation in the Wild: Towards Segmentation of Semi-structured & Unstructured Chats [2.9360071145551068]
  We analyze the capabilities of state-of-the-art topic segmentation models on unstructured texts. We find that pre-training on a large corpus of structured text does not help in transferability to unstructured texts. Training from scratch with only a relatively small-sized dataset of the target unstructured domain improves the segmentation results by a significant margin.
  arXiv  Detail & Related papers  (2022-11-27T22:17:16Z)
• StructToken : Rethinking Semantic Segmentation with Structural Prior [14.056789487558731]
  We present a new paradigm for semantic segmentation, named structure-aware extraction. It generates the segmentation results via the interactions between a set of learned structure tokens and the image feature, which aims to progressively extract the structural information of each category from the feature. Our StructToken outperforms the state-of-the-art on three widely-used benchmarks, including ADE20K, Cityscapes, and COCO-Stuff-10K.
  arXiv  Detail & Related papers  (2022-03-23T17:58:31Z)
• 3D Compositional Zero-shot Learning with DeCompositional Consensus [102.7571947144639]
  We argue that part knowledge should be composable beyond the observed object classes. We present 3D Compositional Zero-shot Learning as a problem of part generalization from seen to unseen object classes.
  arXiv  Detail & Related papers  (2021-11-29T16:34:53Z)
• HS3: Learning with Proper Task Complexity in Hierarchically Supervised Semantic Segmentation [81.87943324048756]
  We propose Hierarchically Supervised Semantic (HS3), a training scheme that supervises intermediate layers in a segmentation network to learn meaningful representations by varying task complexity. Our proposed HS3-Fuse framework further improves segmentation predictions and achieves state-of-the-art results on two large segmentation benchmarks: NYUD-v2 and Cityscapes.
  arXiv  Detail & Related papers  (2021-11-03T16:33:29Z)
• Auto-Panoptic: Cooperative Multi-Component Architecture Search for Panoptic Segmentation [144.50154657257605]
  We propose an efficient framework to simultaneously search for all main components including backbone, segmentation branches, and feature fusion module. Our searched architecture, namely Auto-Panoptic, achieves the new state-of-the-art on the challenging COCO and ADE20K benchmarks.
  arXiv  Detail & Related papers  (2020-10-30T08:34:35Z)
• Bidirectional Graph Reasoning Network for Panoptic Segmentation [126.06251745669107]
  We introduce a Bidirectional Graph Reasoning Network (BGRNet) to mine the intra-modular and intermodular relations within and between foreground things and background stuff classes. BGRNet first constructs image-specific graphs in both instance and semantic segmentation branches that enable flexible reasoning at the proposal level and class level.
  arXiv  Detail & Related papers  (2020-04-14T02:32:10Z)
• Motion-supervised Co-Part Segmentation [88.40393225577088]
  We propose a self-supervised deep learning method for co-part segmentation. Our approach develops the idea that motion information inferred from videos can be leveraged to discover meaningful object parts.
  arXiv  Detail & Related papers  (2020-04-07T09:56:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.