Towards Open-World Segmentation of Parts
- URL: http://arxiv.org/abs/2305.16804v1
- Date: Fri, 26 May 2023 10:34:58 GMT
- Title: Towards Open-World Segmentation of Parts
- Authors: Tai-Yu Pan, Qing Liu, Wei-Lun Chao, Brian Price
- Abstract summary: We propose to explore a class-agnostic part segmentation task.
We argue that models trained without part classes can better localize parts and segment them on objects unseen in training.
We show notable and consistent gains by our approach, essentially a critical step towards open-world part segmentation.
- Score: 16.056921233445784
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Segmenting object parts such as cup handles and animal bodies is important in
many real-world applications but requires more annotation effort. The largest
dataset nowadays contains merely two hundred object categories, implying the
difficulty to scale up part segmentation to an unconstrained setting. To
address this, we propose to explore a seemingly simplified but empirically
useful and scalable task, class-agnostic part segmentation. In this problem, we
disregard the part class labels in training and instead treat all of them as a
single part class. We argue and demonstrate that models trained without part
classes can better localize parts and segment them on objects unseen in
training. We then present two further improvements. First, we propose to make
the model object-aware, leveraging the fact that parts are "compositions",
whose extents are bounded by the corresponding objects and whose appearances
are by nature not independent but bundled. Second, we introduce a novel
approach to improve part segmentation on unseen objects, inspired by an
interesting finding -- for unseen objects, the pixel-wise features extracted by
the model often reveal high-quality part segments. To this end, we propose a
novel self-supervised procedure that iterates between pixel clustering and
supervised contrastive learning that pulls pixels closer or pushes them away.
Via extensive experiments on PartImageNet and Pascal-Part, we show notable and
consistent gains by our approach, essentially a critical step towards
open-world part segmentation.
Related papers
- Self-Supervised Instance Segmentation by Grasping [84.2469669256257]
We learn a grasp segmentation model to segment the grasped object from before and after grasp images.
Using the segmented objects, we can "cut" objects from their original scenes and "paste" them into new scenes to generate instance supervision.
We show that our grasp segmentation model provides a 5x error reduction when segmenting grasped objects compared with traditional image subtraction approaches.
arXiv Detail & Related papers (2023-05-10T16:51:36Z) - Self-Supervised Learning of Object Parts for Semantic Segmentation [7.99536002595393]
We argue that self-supervised learning of object parts is a solution to this issue.
Our method surpasses the state-of-the-art on three semantic segmentation benchmarks by 17%-3%.
arXiv Detail & Related papers (2022-04-27T17:55:17Z) - Towards Robust Part-aware Instance Segmentation for Industrial Bin
Picking [113.79582950811348]
We formulate a novel part-aware instance segmentation pipeline for industrial bin picking.
We design a part-aware network to predict part masks and part-to-part offsets, followed by a part aggregation module to assemble the recognized parts into instances.
We contribute the first instance segmentation dataset, which contains a variety of industrial objects that are thin and have non-trivial shapes.
arXiv Detail & Related papers (2022-03-05T14:58:05Z) - Learning with Free Object Segments for Long-Tailed Instance Segmentation [15.563842274862314]
We find that an abundance of instance segments can potentially be obtained freely from object-centric im-ages.
Motivated by these insights, we propose FreeSeg for extracting and leveraging these "free" object segments.
FreeSeg achieves state-of-the-art accuracy for segmenting rare object categories.
arXiv Detail & Related papers (2022-02-22T19:06:16Z) - 3D Compositional Zero-shot Learning with DeCompositional Consensus [102.7571947144639]
We argue that part knowledge should be composable beyond the observed object classes.
We present 3D Compositional Zero-shot Learning as a problem of part generalization from seen to unseen object classes.
arXiv Detail & Related papers (2021-11-29T16:34:53Z) - The Emergence of Objectness: Learning Zero-Shot Segmentation from Videos [59.12750806239545]
We show that a video has different views of the same scene related by moving components, and the right region segmentation and region flow would allow mutual view synthesis.
Our model starts with two separate pathways: an appearance pathway that outputs feature-based region segmentation for a single image, and a motion pathway that outputs motion features for a pair of images.
By training the model to minimize view synthesis errors based on segment flow, our appearance and motion pathways learn region segmentation and flow estimation automatically without building them up from low-level edges or optical flows respectively.
arXiv Detail & Related papers (2021-11-11T18:59:11Z) - Unsupervised Part Discovery from Contrastive Reconstruction [90.88501867321573]
The goal of self-supervised visual representation learning is to learn strong, transferable image representations.
We propose an unsupervised approach to object part discovery and segmentation.
Our method yields semantic parts consistent across fine-grained but visually distinct categories.
arXiv Detail & Related papers (2021-11-11T17:59:42Z) - Unsupervised Co-part Segmentation through Assembly [42.874278526843305]
We propose an unsupervised learning approach for co-part segmentation from images.
We leverage motion information embedded in videos and explicitly extract latent representations to segment meaningful object parts.
We show that our approach can achieve meaningful and compact part segmentation, outperforming state-of-the-art approaches on diverse benchmarks.
arXiv Detail & Related papers (2021-06-10T16:22:53Z) - Unsupervised Part Discovery by Unsupervised Disentanglement [10.664434993386525]
Part segmentations provide information about part localizations on the level of individual pixels.
Large annotation costs limit the scalability of supervised algorithms to other object categories.
Our work demonstrates the feasibility to discover semantic part segmentations without supervision.
arXiv Detail & Related papers (2020-09-09T12:34:37Z) - Part-aware Prototype Network for Few-shot Semantic Segmentation [50.581647306020095]
We propose a novel few-shot semantic segmentation framework based on the prototype representation.
Our key idea is to decompose the holistic class representation into a set of part-aware prototypes.
We develop a novel graph neural network model to generate and enhance the proposed part-aware prototypes.
arXiv Detail & Related papers (2020-07-13T11:03:09Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.