A Unified Query-based Paradigm for Camouflaged Instance Segmentation
- URL: http://arxiv.org/abs/2308.07392v2
- Date: Tue, 29 Aug 2023 12:37:04 GMT
- Title: A Unified Query-based Paradigm for Camouflaged Instance Segmentation
- Authors: Bo Dong, Jialun Pei, Rongrong Gao, Tian-Zhu Xiang, Shuo Wang, Huan
Xiong
- Abstract summary: We propose a unified query-based multi-task learning framework for camouflaged instance segmentation, termed UQFormer.
Our model views the instance segmentation as a query-based direct set prediction problem, without other post-processing such as non-maximal suppression.
Compared with 14 state-of-the-art approaches, our UQFormer significantly improves the performance of camouflaged instance segmentation.
- Score: 26.91533966120182
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Due to the high similarity between camouflaged instances and the background,
the recently proposed camouflaged instance segmentation (CIS) faces challenges
in accurate localization and instance segmentation. To this end, inspired by
query-based transformers, we propose a unified query-based multi-task learning
framework for camouflaged instance segmentation, termed UQFormer, which builds
a set of mask queries and a set of boundary queries to learn a shared composed
query representation and efficiently integrates global camouflaged object
region and boundary cues, for simultaneous instance segmentation and instance
boundary detection in camouflaged scenarios. Specifically, we design a composed
query learning paradigm that learns a shared representation to capture object
region and boundary features by the cross-attention interaction of mask queries
and boundary queries in the designed multi-scale unified learning transformer
decoder. Then, we present a transformer-based multi-task learning framework for
simultaneous camouflaged instance segmentation and camouflaged instance
boundary detection based on the learned composed query representation, which
also forces the model to learn a strong instance-level query representation.
Notably, our model views the instance segmentation as a query-based direct set
prediction problem, without other post-processing such as non-maximal
suppression. Compared with 14 state-of-the-art approaches, our UQFormer
significantly improves the performance of camouflaged instance segmentation.
Our code will be available at https://github.com/dongbo811/UQFormer.
Related papers
- DQFormer: Towards Unified LiDAR Panoptic Segmentation with Decoupled Queries [14.435906383301555]
We propose a novel framework dubbed DQFormer to implement semantic and instance segmentation in a unified workflow.
Specifically, we design a decoupled query generator to propose informative queries with semantics by localizing things/stuff positions.
We also introduce a query-oriented mask decoder to decode corresponding segmentation masks.
arXiv Detail & Related papers (2024-08-28T14:14:33Z) - Temporal-aware Hierarchical Mask Classification for Video Semantic
Segmentation [62.275143240798236]
Video semantic segmentation dataset has limited categories per video.
Less than 10% of queries could be matched to receive meaningful gradient updates during VSS training.
Our method achieves state-of-the-art performance on the latest challenging VSS benchmark VSPW without bells and whistles.
arXiv Detail & Related papers (2023-09-14T20:31:06Z) - Learning Equivariant Segmentation with Instance-Unique Querying [47.52528819153683]
We devise a new training framework that boosts query-based models through discriminative query embedding learning.
Our algorithm uses the queries to retrieve the corresponding instances from the whole training dataset.
On top of four famous, query-based models, our training algorithm provides significant performance gains.
arXiv Detail & Related papers (2022-10-03T13:14:00Z) - Semantic Attention and Scale Complementary Network for Instance
Segmentation in Remote Sensing Images [54.08240004593062]
We propose an end-to-end multi-category instance segmentation model, which consists of a Semantic Attention (SEA) module and a Scale Complementary Mask Branch (SCMB)
SEA module contains a simple fully convolutional semantic segmentation branch with extra supervision to strengthen the activation of interest instances on the feature map.
SCMB extends the original single mask branch to trident mask branches and introduces complementary mask supervision at different scales.
arXiv Detail & Related papers (2021-07-25T08:53:59Z) - SOLO: A Simple Framework for Instance Segmentation [84.00519148562606]
"instance categories" assigns categories to each pixel within an instance according to the instance's location.
"SOLO" is a simple, direct, and fast framework for instance segmentation with strong performance.
Our approach achieves state-of-the-art results for instance segmentation in terms of both speed and accuracy.
arXiv Detail & Related papers (2021-06-30T09:56:54Z) - SOLQ: Segmenting Objects by Learning Queries [33.02115826341877]
In SOLQ, each query represents one object and has multiple representations: class, location and mask.
SOLQ can achieve state-of-the-art performance, surpassing most of existing approaches.
Joint learning of unified query representation can greatly improve the detection performance of original DETR.
arXiv Detail & Related papers (2021-06-04T09:03:31Z) - QueryInst: Parallelly Supervised Mask Query for Instance Segmentation [53.5613957875507]
We present QueryInst, a query based instance segmentation method driven by parallel supervision on dynamic mask heads.
We conduct extensive experiments on three challenging benchmarks, i.e., COCO, CityScapes, and YouTube-VIS.
QueryInst achieves the best performance among all online VIS approaches and strikes a decent speed-accuracy trade-off.
arXiv Detail & Related papers (2021-05-05T08:38:25Z) - The Devil is in the Boundary: Exploiting Boundary Representation for
Basis-based Instance Segmentation [85.153426159438]
We propose Basis based Instance(B2Inst) to learn a global boundary representation that can complement existing global-mask-based methods.
Our B2Inst leads to consistent improvements and accurately parses out the instance boundaries in a scene.
arXiv Detail & Related papers (2020-11-26T11:26:06Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.