Learning with Free Object Segments for Long-Tailed Instance Segmentation

• URL: http://arxiv.org/abs/2202.11124v1
• Date: Tue, 22 Feb 2022 19:06:16 GMT
• Title: Learning with Free Object Segments for Long-Tailed Instance Segmentation
• Authors: Cheng Zhang, Tai-Yu Pan, Tianle Chen, Jike Zhong, Wenjin Fu, Wei-Lun Chao
• Abstract summary: We find that an abundance of instance segments can potentially be obtained freely from object-centric im-ages. Motivated by these insights, we propose FreeSeg for extracting and leveraging these "free" object segments. FreeSeg achieves state-of-the-art accuracy for segmenting rare object categories.
• Score: 15.563842274862314
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: One fundamental challenge in building an instance segmentation model for a large number of classes in complex scenes is the lack of training examples, especially for rare objects. In this paper, we explore the possibility to increase the training examples without laborious data collection and annotation. We find that an abundance of instance segments can potentially be obtained freely from object-centric im-ages, according to two insights: (i) an object-centric image usually contains one salient object in a simple background; (ii) objects from the same class often share similar appearances or similar contrasts to the background. Motivated by these insights, we propose a simple and scalable framework FreeSeg for extracting and leveraging these "free" object foreground segments to facilitate model training in long-tailed instance segmentation. Concretely, we employ off-the-shelf object foreground extraction techniques (e.g., image co-segmentation) to generate instance mask candidates, followed by segments refinement and ranking. The resulting high-quality object segments can be used to augment the existing long-tailed dataset, e.g., by copying and pasting the segments onto the original training images. On the LVIS benchmark, we show that FreeSeg yields substantial improvements on top of strong baselines and achieves state-of-the-art accuracy for segmenting rare object categories.

Related papers

• Synthetic Instance Segmentation from Semantic Image Segmentation Masks [15.477053085267404]
  We propose a novel paradigm called Synthetic Instance (SISeg) SISeg instance segmentation results by leveraging image masks generated by existing semantic segmentation models. In other words, the proposed model does not need extra manpower or higher computational expenses.
  arXiv  Detail & Related papers  (2023-08-02T05:13:02Z)
• Semantic-SAM: Segment and Recognize Anything at Any Granularity [83.64686655044765]
  We introduce Semantic-SAM, a universal image segmentation model to enable segment and recognize anything at any desired granularity. We consolidate multiple datasets across three granularities and introduce decoupled classification for objects and parts. For the multi-granularity capability, we propose a multi-choice learning scheme during training, enabling each click to generate masks at multiple levels.
  arXiv  Detail & Related papers  (2023-07-10T17:59:40Z)
• Towards Open-World Segmentation of Parts [16.056921233445784]
  We propose to explore a class-agnostic part segmentation task. We argue that models trained without part classes can better localize parts and segment them on objects unseen in training. We show notable and consistent gains by our approach, essentially a critical step towards open-world part segmentation.
  arXiv  Detail & Related papers  (2023-05-26T10:34:58Z)
• Weakly-Supervised Concealed Object Segmentation with SAM-based Pseudo Labeling and Multi-scale Feature Grouping [40.07070188661184]
  Weakly-Supervised Concealed Object (WSCOS) aims to segment objects well blended with surrounding environments. It is hard to distinguish concealed objects from the background due to the intrinsic similarity. We propose a new WSCOS method to address these two challenges.
  arXiv  Detail & Related papers  (2023-05-18T14:31:34Z)
• Self-Supervised Instance Segmentation by Grasping [84.2469669256257]
  We learn a grasp segmentation model to segment the grasped object from before and after grasp images. Using the segmented objects, we can "cut" objects from their original scenes and "paste" them into new scenes to generate instance supervision. We show that our grasp segmentation model provides a 5x error reduction when segmenting grasped objects compared with traditional image subtraction approaches.
  arXiv  Detail & Related papers  (2023-05-10T16:51:36Z)
• Leveraging GAN Priors for Few-Shot Part Segmentation [43.35150430895919]
  Few-shot part segmentation aims to separate different parts of an object given only a few samples. We propose to learn task-specific features in a "pre-training"-"fine-tuning" paradigm.
  arXiv  Detail & Related papers  (2022-07-27T10:17:07Z)
• Open-world Semantic Segmentation via Contrasting and Clustering Vision-Language Embedding [95.78002228538841]
  We propose a new open-world semantic segmentation pipeline that makes the first attempt to learn to segment semantic objects of various open-world categories without any efforts on dense annotations. Our method can directly segment objects of arbitrary categories, outperforming zero-shot segmentation methods that require data labeling on three benchmark datasets.
  arXiv  Detail & Related papers  (2022-07-18T09:20:04Z)
• Towards Robust Part-aware Instance Segmentation for Industrial Bin Picking [113.79582950811348]
  We formulate a novel part-aware instance segmentation pipeline for industrial bin picking. We design a part-aware network to predict part masks and part-to-part offsets, followed by a part aggregation module to assemble the recognized parts into instances. We contribute the first instance segmentation dataset, which contains a variety of industrial objects that are thin and have non-trivial shapes.
  arXiv  Detail & Related papers  (2022-03-05T14:58:05Z)
• SCNet: Enhancing Few-Shot Semantic Segmentation by Self-Contrastive Background Prototypes [56.387647750094466]
  Few-shot semantic segmentation aims to segment novel-class objects in a query image with only a few annotated examples. Most of advanced solutions exploit a metric learning framework that performs segmentation through matching each pixel to a learned foreground prototype. This framework suffers from biased classification due to incomplete construction of sample pairs with the foreground prototype only.
  arXiv  Detail & Related papers  (2021-04-19T11:21:47Z)
• Part-aware Prototype Network for Few-shot Semantic Segmentation [50.581647306020095]
  We propose a novel few-shot semantic segmentation framework based on the prototype representation. Our key idea is to decompose the holistic class representation into a set of part-aware prototypes. We develop a novel graph neural network model to generate and enhance the proposed part-aware prototypes.
  arXiv  Detail & Related papers  (2020-07-13T11:03:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.