An Efficient Instance Segmentation Framework Based on Oriented Bounding Boxes
- URL: http://arxiv.org/abs/2401.08174v3
- Date: Mon, 1 Jul 2024 15:16:02 GMT
- Title: An Efficient Instance Segmentation Framework Based on Oriented Bounding Boxes
- Authors: Zhen Zhou, Junfeng Fan, Yunkai Ma, Sihan Zhao, Fengshui Jing, Min Tan,
- Abstract summary: This paper proposes a unified coarse-to-fine instance segmentation framework, CFNet.
It uses box prompt-based segmentation foundation models (BSMs), e.g., Segment Anything Model.
Experiments demonstrate that CFNet outperforms current instance segmentation methods on both industrial and public datasets.
- Score: 6.582483085163108
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Instance segmentation for completely occluded objects and dense objects in robot vision measurement are two challenging tasks. To uniformly deal with them, this paper proposes a unified coarse-to-fine instance segmentation framework, CFNet, which uses box prompt-based segmentation foundation models (BSMs), e.g., Segment Anything Model. Specifically, CFNet first detects oriented bounding boxes (OBBs) to distinguish instances and provide coarse localization information. Then, it predicts OBB prompt-related masks for fine segmentation. CFNet performs instance segmentation with OBBs that only contain partial object boundaries on occluders to predict occluded object instances, which overcomes the difficulty of existing amodal instance segmentation methods in directly predicting occluded objects. In addition, since OBBs only serve as prompts, CFNet alleviates the over-dependence on bounding box detection performance of current instance segmentation methods using OBBs for dense objects. Moreover, to enable BSMs to handle OBB prompts, we propose a novel OBB prompt encoder. To make CFNet more lightweight, we perform knowledge distillation on it and introduce a Gaussian label smoothing method for teacher model outputs. Experiments demonstrate that CFNet outperforms current instance segmentation methods on both industrial and public datasets. The code is available at https://github.com/zhen6618/OBBInstanceSegmentation.
Related papers
- BAISeg: Boundary Assisted Weakly Supervised Instance Segmentation [9.6046915661065]
How to extract instance-level masks without instance-level supervision is the main challenge of weakly supervised instance segmentation (WSIS)
Popular WSIS methods estimate a displacement field (DF) via learning inter-pixel relations and perform clustering to identify instances.
We propose Boundary-Assisted Instance (BAISeg), which is a novel paradigm for WSIS that realizes instance segmentation with pixel-level annotations.
arXiv Detail & Related papers (2024-05-27T15:14:09Z) - Look Before You Match: Instance Understanding Matters in Video Object
Segmentation [114.57723592870097]
In this paper, we argue that instance matters in video object segmentation (VOS)
We present a two-branch network for VOS, where the query-based instance segmentation (IS) branch delves into the instance details of the current frame and the VOS branch performs spatial-temporal matching with the memory bank.
We employ well-learned object queries from IS branch to inject instance-specific information into the query key, with which the instance-auged matching is further performed.
arXiv Detail & Related papers (2022-12-13T18:59:59Z) - Sparse Instance Activation for Real-Time Instance Segmentation [72.23597664935684]
We propose a conceptually novel, efficient, and fully convolutional framework for real-time instance segmentation.
SparseInst has extremely fast inference speed and achieves 40 FPS and 37.9 AP on the COCO benchmark.
arXiv Detail & Related papers (2022-03-24T03:15:39Z) - FreeSOLO: Learning to Segment Objects without Annotations [191.82134817449528]
We present FreeSOLO, a self-supervised instance segmentation framework built on top of the simple instance segmentation method SOLO.
Our method also presents a novel localization-aware pre-training framework, where objects can be discovered from complicated scenes in an unsupervised manner.
arXiv Detail & Related papers (2022-02-24T16:31:44Z) - SOLO: A Simple Framework for Instance Segmentation [84.00519148562606]
"instance categories" assigns categories to each pixel within an instance according to the instance's location.
"SOLO" is a simple, direct, and fast framework for instance segmentation with strong performance.
Our approach achieves state-of-the-art results for instance segmentation in terms of both speed and accuracy.
arXiv Detail & Related papers (2021-06-30T09:56:54Z) - Object-Guided Instance Segmentation With Auxiliary Feature Refinement
for Biological Images [58.914034295184685]
Instance segmentation is of great importance for many biological applications, such as study of neural cell interactions, plant phenotyping, and quantitatively measuring how cells react to drug treatment.
Box-based instance segmentation methods capture objects via bounding boxes and then perform individual segmentation within each bounding box region.
Our method first detects the center points of the objects, from which the bounding box parameters are then predicted.
The segmentation branch reuses the object features as guidance to separate target object from the neighboring ones within the same bounding box region.
arXiv Detail & Related papers (2021-06-14T04:35:36Z) - Target-Aware Object Discovery and Association for Unsupervised Video
Multi-Object Segmentation [79.6596425920849]
This paper addresses the task of unsupervised video multi-object segmentation.
We introduce a novel approach for more accurate and efficient unseen-temporal segmentation.
We evaluate the proposed approach on DAVIS$_17$ and YouTube-VIS, and the results demonstrate that it outperforms state-of-the-art methods both in segmentation accuracy and inference speed.
arXiv Detail & Related papers (2021-04-10T14:39:44Z) - Deep Variational Instance Segmentation [7.334808870313923]
State-of-the-art algorithms often employ two separate stages, the first one generating object proposals and the second one recognizing and refining the boundaries.
We propose a novel algorithm that directly utilizes a fully convolutional network (FCN) to predict instance labels.
arXiv Detail & Related papers (2020-07-22T17:57:49Z) - Towards Bounding-Box Free Panoptic Segmentation [16.4548904544277]
We introduce a new Bounding-Box Free Network (BBFNet) for panoptic segmentation.
BBFNet predicts coarse watershed levels and uses them to detect large instance candidates where boundaries are well defined.
For smaller instances, whose boundaries are less reliable, BBFNet also predicts instance centers by means of Hough voting followed by mean-shift to reliably detect small objects.
arXiv Detail & Related papers (2020-02-18T16:34:01Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.