Box2Mask: Box-supervised Instance Segmentation via Level-set Evolution
- URL: http://arxiv.org/abs/2212.01579v1
- Date: Sat, 3 Dec 2022 09:32:14 GMT
- Title: Box2Mask: Box-supervised Instance Segmentation via Level-set Evolution
- Authors: Wentong Li, Wenyu Liu, Jianke Zhu, Miaomiao Cui, Risheng Yu, Xiansheng
Hua, Lei Zhang
- Abstract summary: This paper presents a novel single-shot instance segmentation approach, namely Box2Mask.
Box2Mask integrates the classical level-set evolution model into deep neural network learning to achieve accurate mask prediction with only bounding box supervision.
- Score: 38.88010537144528
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In contrast to fully supervised methods using pixel-wise mask labels,
box-supervised instance segmentation takes advantage of simple box annotations,
which has recently attracted increasing research attention. This paper presents
a novel single-shot instance segmentation approach, namely Box2Mask, which
integrates the classical level-set evolution model into deep neural network
learning to achieve accurate mask prediction with only bounding box
supervision. Specifically, both the input image and its deep features are
employed to evolve the level-set curves implicitly, and a local consistency
module based on a pixel affinity kernel is used to mine the local context and
spatial relations. Two types of single-stage frameworks, i.e., CNN-based and
transformer-based frameworks, are developed to empower the level-set evolution
for box-supervised instance segmentation, and each framework consists of three
essential components: instance-aware decoder, box-level matching assignment and
level-set evolution. By minimizing the level-set energy function, the mask map
of each instance can be iteratively optimized within its bounding box
annotation. The experimental results on five challenging testbeds, covering
general scenes, remote sensing, medical and scene text images, demonstrate the
outstanding performance of our proposed Box2Mask approach for box-supervised
instance segmentation. In particular, with the Swin-Transformer large backbone,
our Box2Mask obtains 42.4% mask AP on COCO, which is on par with the recently
developed fully mask-supervised methods. The code is available at:
https://github.com/LiWentomng/boxlevelset.
Related papers
- Skeleton-Guided Instance Separation for Fine-Grained Segmentation in
Microscopy [23.848474219551818]
One of the fundamental challenges in microscopy (MS) image analysis is instance segmentation (IS)
We propose a novel one-stage framework named A2B-IS to address this challenge and enhance the accuracy of IS in MS images.
Our method has been thoroughly validated on two large-scale MS datasets.
arXiv Detail & Related papers (2024-01-18T11:14:32Z) - BoxSnake: Polygonal Instance Segmentation with Box Supervision [34.487089567665556]
We propose a new end-to-end training technique, termed BoxSnake, to achieve effective polygonal instance segmentation using only box annotations for the first time.
Compared with the mask-based weakly-supervised methods, BoxSnake further reduces the performance gap between the predicted segmentation and the bounding box, and shows significant superiority on the Cityscapes dataset.
arXiv Detail & Related papers (2023-03-21T06:54:18Z) - A Simple Framework for Open-Vocabulary Segmentation and Detection [85.21641508535679]
We present OpenSeeD, a simple Open-vocabulary and Detection framework that jointly learns from different segmentation and detection datasets.
We first introduce a pre-trained text encoder to encode all the visual concepts in two tasks and learn a common semantic space for them.
After pre-training, our model exhibits competitive or stronger zero-shot transferability for both segmentation and detection.
arXiv Detail & Related papers (2023-03-14T17:58:34Z) - Box-supervised Instance Segmentation with Level Set Evolution [41.19797478617953]
We propose a box-supervised instance segmentation approach, which integrates the classical level set model with deep neural network delicately.
A simple mask supervised SOLOv2 model is adapted to predict the instance-aware mask map as the level set for each instance.
The experimental results on four challenging benchmarks demonstrate the leading performance of our proposed approach.
arXiv Detail & Related papers (2022-07-19T03:59:44Z) - Accelerating the creation of instance segmentation training sets through
bounding box annotation [25.85927871251385]
Our work proposes to delineate instances in three steps, based on a semi-automatic approach.
The sole definition of extreme points results in a model accuracy that would require up to 10 times more resources if the masks were defined through fully manual delineation.
arXiv Detail & Related papers (2022-05-23T18:37:03Z) - Point Cloud Instance Segmentation with Semi-supervised Bounding-Box
Mining [17.69745159912481]
We introduce the first semi-supervised point cloud instance segmentation framework (SPIB) using both labeled and unlabelled bounding boxes as supervision.
Our method can achieve competitive performance compared with the recent fully-supervised methods.
arXiv Detail & Related papers (2021-11-30T08:40:40Z) - Box-Adapt: Domain-Adaptive Medical Image Segmentation using Bounding
BoxSupervision [52.45336255472669]
We propose a weakly supervised do-main adaptation setting for deep learning.
Box-Adapt fully explores the fine-grained segmenta-tion mask in the source domain and the weak bounding box in the target domain.
We demonstrate the effectiveness of our method in the liver segmentation task.
arXiv Detail & Related papers (2021-08-19T01:51:04Z) - The Devil is in the Boundary: Exploiting Boundary Representation for
Basis-based Instance Segmentation [85.153426159438]
We propose Basis based Instance(B2Inst) to learn a global boundary representation that can complement existing global-mask-based methods.
Our B2Inst leads to consistent improvements and accurately parses out the instance boundaries in a scene.
arXiv Detail & Related papers (2020-11-26T11:26:06Z) - PointINS: Point-based Instance Segmentation [117.38579097923052]
Mask representation in instance segmentation with Point-of-Interest (PoI) features is challenging because learning a high-dimensional mask feature for each instance requires a heavy computing burden.
We propose an instance-aware convolution, which decomposes this mask representation learning task into two tractable modules.
Along with instance-aware convolution, we propose PointINS, a simple and practical instance segmentation approach.
arXiv Detail & Related papers (2020-03-13T08:24:58Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.