An Efficient Instance Segmentation Framework Based on Oriented Bounding Boxes

• URL: http://arxiv.org/abs/2401.08174v3
• Date: Mon, 1 Jul 2024 15:16:02 GMT
• Title: An Efficient Instance Segmentation Framework Based on Oriented Bounding Boxes
• Authors: Zhen Zhou, Junfeng Fan, Yunkai Ma, Sihan Zhao, Fengshui Jing, Min Tan,
• Abstract summary: This paper proposes a unified coarse-to-fine instance segmentation framework, CFNet. It uses box prompt-based segmentation foundation models (BSMs), e.g., Segment Anything Model. Experiments demonstrate that CFNet outperforms current instance segmentation methods on both industrial and public datasets.
• Score: 6.582483085163108
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Instance segmentation for completely occluded objects and dense objects in robot vision measurement are two challenging tasks. To uniformly deal with them, this paper proposes a unified coarse-to-fine instance segmentation framework, CFNet, which uses box prompt-based segmentation foundation models (BSMs), e.g., Segment Anything Model. Specifically, CFNet first detects oriented bounding boxes (OBBs) to distinguish instances and provide coarse localization information. Then, it predicts OBB prompt-related masks for fine segmentation. CFNet performs instance segmentation with OBBs that only contain partial object boundaries on occluders to predict occluded object instances, which overcomes the difficulty of existing amodal instance segmentation methods in directly predicting occluded objects. In addition, since OBBs only serve as prompts, CFNet alleviates the over-dependence on bounding box detection performance of current instance segmentation methods using OBBs for dense objects. Moreover, to enable BSMs to handle OBB prompts, we propose a novel OBB prompt encoder. To make CFNet more lightweight, we perform knowledge distillation on it and introduce a Gaussian label smoothing method for teacher model outputs. Experiments demonstrate that CFNet outperforms current instance segmentation methods on both industrial and public datasets. The code is available at https://github.com/zhen6618/OBBInstanceSegmentation.

Related papers

• BAISeg: Boundary Assisted Weakly Supervised Instance Segmentation [9.6046915661065]
  How to extract instance-level masks without instance-level supervision is the main challenge of weakly supervised instance segmentation (WSIS) Popular WSIS methods estimate a displacement field (DF) via learning inter-pixel relations and perform clustering to identify instances. We propose Boundary-Assisted Instance (BAISeg), which is a novel paradigm for WSIS that realizes instance segmentation with pixel-level annotations.
  arXiv  Detail & Related papers  (2024-05-27T15:14:09Z)
• Efficient Temporal Action Segmentation via Boundary-aware Query Voting [51.92693641176378]
  BaFormer is a boundary-aware Transformer network that tokenizes each video segment as an instance token. BaFormer significantly reduces the computational costs, utilizing only 6% of the running time.
  arXiv  Detail & Related papers  (2024-05-25T00:44:13Z)
• ElC-OIS: Ellipsoidal Clustering for Open-World Instance Segmentation on LiDAR Data [13.978966783993146]
  Open-world Instance (OIS) is a challenging task that aims to accurately segment every object instance appearing in the current observation. This is important for safety-critical applications such as robust autonomous navigation. We present a flexible and effective OIS framework for LiDAR point cloud that can accurately segment both known and unknown instances.
  arXiv  Detail & Related papers  (2023-03-08T03:22:11Z)
• Look Before You Match: Instance Understanding Matters in Video Object Segmentation [114.57723592870097]
  In this paper, we argue that instance matters in video object segmentation (VOS) We present a two-branch network for VOS, where the query-based instance segmentation (IS) branch delves into the instance details of the current frame and the VOS branch performs spatial-temporal matching with the memory bank. We employ well-learned object queries from IS branch to inject instance-specific information into the query key, with which the instance-auged matching is further performed.
  arXiv  Detail & Related papers  (2022-12-13T18:59:59Z)
• Sparse Instance Activation for Real-Time Instance Segmentation [72.23597664935684]
  We propose a conceptually novel, efficient, and fully convolutional framework for real-time instance segmentation. SparseInst has extremely fast inference speed and achieves 40 FPS and 37.9 AP on the COCO benchmark.
  arXiv  Detail & Related papers  (2022-03-24T03:15:39Z)
• Boosting Weakly Supervised Object Detection via Learning Bounding Box Adjusters [76.36104006511684]
  Weakly-supervised object detection (WSOD) has emerged as an inspiring recent topic to avoid expensive instance-level object annotations. We defend the problem setting for improving localization performance by leveraging the bounding box regression knowledge from a well-annotated auxiliary dataset. Our method performs favorably against state-of-the-art WSOD methods and knowledge transfer model with similar problem setting.
  arXiv  Detail & Related papers  (2021-08-03T13:38:20Z)
• SOLO: A Simple Framework for Instance Segmentation [84.00519148562606]
  "instance categories" assigns categories to each pixel within an instance according to the instance's location. "SOLO" is a simple, direct, and fast framework for instance segmentation with strong performance. Our approach achieves state-of-the-art results for instance segmentation in terms of both speed and accuracy.
  arXiv  Detail & Related papers  (2021-06-30T09:56:54Z)
• Object-Guided Instance Segmentation With Auxiliary Feature Refinement for Biological Images [58.914034295184685]
  Instance segmentation is of great importance for many biological applications, such as study of neural cell interactions, plant phenotyping, and quantitatively measuring how cells react to drug treatment. Box-based instance segmentation methods capture objects via bounding boxes and then perform individual segmentation within each bounding box region. Our method first detects the center points of the objects, from which the bounding box parameters are then predicted. The segmentation branch reuses the object features as guidance to separate target object from the neighboring ones within the same bounding box region.
  arXiv  Detail & Related papers  (2021-06-14T04:35:36Z)
• BBAM: Bounding Box Attribution Map for Weakly Supervised Semantic and Instance Segmentation [19.55647093153416]
  Weakly supervised segmentation methods using bounding box annotations focus on obtaining a pixel-level mask from each box containing an object. In this work, we utilize higher-level information from the behavior of a trained object detector, by seeking the smallest areas of the image from which the object detector produces almost the same result as it does from the whole image. These areas constitute a bounding-box attribution map (BBAM), which identifies the target object in its bounding box and thus serves as pseudo ground-truth for weakly supervised semantic and COCO instance segmentation.
  arXiv  Detail & Related papers  (2021-03-16T08:29:33Z)
• Towards Bounding-Box Free Panoptic Segmentation [16.4548904544277]
  We introduce a new Bounding-Box Free Network (BBFNet) for panoptic segmentation. BBFNet predicts coarse watershed levels and uses them to detect large instance candidates where boundaries are well defined. For smaller instances, whose boundaries are less reliable, BBFNet also predicts instance centers by means of Hough voting followed by mean-shift to reliably detect small objects.
  arXiv  Detail & Related papers  (2020-02-18T16:34:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.