Learning Equivariant Segmentation with Instance-Unique Querying
- URL: http://arxiv.org/abs/2210.00911v1
- Date: Mon, 3 Oct 2022 13:14:00 GMT
- Title: Learning Equivariant Segmentation with Instance-Unique Querying
- Authors: Wenguan Wang, James Liang, Dongfang Liu
- Abstract summary: We devise a new training framework that boosts query-based models through discriminative query embedding learning.
Our algorithm uses the queries to retrieve the corresponding instances from the whole training dataset.
On top of four famous, query-based models, our training algorithm provides significant performance gains.
- Score: 47.52528819153683
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Prevalent state-of-the-art instance segmentation methods fall into a
query-based scheme, in which instance masks are derived by querying the image
feature using a set of instance-aware embeddings. In this work, we devise a new
training framework that boosts query-based models through discriminative query
embedding learning. It explores two essential properties, namely dataset-level
uniqueness and transformation equivariance, of the relation between queries and
instances. First, our algorithm uses the queries to retrieve the corresponding
instances from the whole training dataset, instead of only searching within
individual scenes. As querying instances across scenes is more challenging, the
segmenters are forced to learn more discriminative queries for effective
instance separation. Second, our algorithm encourages both image (instance)
representations and queries to be equivariant against geometric
transformations, leading to more robust, instance-query matching. On top of
four famous, query-based models ($i.e.,$ CondInst, SOLOv2, SOTR, and
Mask2Former), our training algorithm provides significant performance gains
($e.g.,$ +1.6 - 3.2 AP) on COCO dataset. In addition, our algorithm promotes
the performance of SOLOv2 by 2.7 AP, on LVISv1 dataset.
Related papers
- FastGAS: Fast Graph-based Annotation Selection for In-Context Learning [53.17606395275021]
In-context learning (ICL) empowers large language models (LLMs) to tackle new tasks by using a series of training instances as prompts.
Existing methods have proposed to select a subset of unlabeled examples for annotation.
We propose a graph-based selection method, FastGAS, designed to efficiently identify high-quality instances.
arXiv Detail & Related papers (2024-06-06T04:05:54Z) - LaSagnA: Language-based Segmentation Assistant for Complex Queries [39.620806493454616]
Large Language Models for Vision (vLLMs) generate detailed perceptual outcomes, including bounding boxes and masks.
In this study, we acknowledge that the main cause of these problems is the insufficient complexity of training queries.
We present three novel strategies to effectively handle the challenges arising from the direct integration of the proposed format.
arXiv Detail & Related papers (2024-04-12T14:40:45Z) - A Unified Query-based Paradigm for Camouflaged Instance Segmentation [26.91533966120182]
We propose a unified query-based multi-task learning framework for camouflaged instance segmentation, termed UQFormer.
Our model views the instance segmentation as a query-based direct set prediction problem, without other post-processing such as non-maximal suppression.
Compared with 14 state-of-the-art approaches, our UQFormer significantly improves the performance of camouflaged instance segmentation.
arXiv Detail & Related papers (2023-08-14T18:23:18Z) - Learning Dynamic Query Combinations for Transformer-based Object
Detection and Segmentation [37.24532930188581]
Transformer-based detection and segmentation methods use a list of learned detection queries to retrieve information from the transformer network.
We empirically find that random convex combinations of the learned queries are still good for the corresponding models.
We propose to learn a convex combination with dynamic coefficients based on the high-level semantics of the image.
arXiv Detail & Related papers (2023-07-23T06:26:27Z) - Instance Image Retrieval by Learning Purely From Within the Dataset [26.59077238274763]
This work looks into a different approach which has not been well investigated for instance image retrieval previously.
By adding an object proposal generator to generate image regions for self-supervised learning, the investigated approach can successfully learn feature representation specific to a given dataset for retrieval.
As experimentally validated, such a simple self-supervised learning + self-boosting'' approach can well compete with the relevant state-of-the-art retrieval methods.
arXiv Detail & Related papers (2022-08-12T04:51:53Z) - Parallel Instance Query Network for Named Entity Recognition [73.30174490672647]
Named entity recognition (NER) is a fundamental task in natural language processing.
Recent works treat named entity recognition as a reading comprehension task, constructing type-specific queries manually to extract entities.
We propose Parallel Instance Query Network (PIQN), which sets up global and learnable instance queries to extract entities in a parallel manner.
arXiv Detail & Related papers (2022-03-20T13:01:25Z) - SOLO: A Simple Framework for Instance Segmentation [84.00519148562606]
"instance categories" assigns categories to each pixel within an instance according to the instance's location.
"SOLO" is a simple, direct, and fast framework for instance segmentation with strong performance.
Our approach achieves state-of-the-art results for instance segmentation in terms of both speed and accuracy.
arXiv Detail & Related papers (2021-06-30T09:56:54Z) - QueryInst: Parallelly Supervised Mask Query for Instance Segmentation [53.5613957875507]
We present QueryInst, a query based instance segmentation method driven by parallel supervision on dynamic mask heads.
We conduct extensive experiments on three challenging benchmarks, i.e., COCO, CityScapes, and YouTube-VIS.
QueryInst achieves the best performance among all online VIS approaches and strikes a decent speed-accuracy trade-off.
arXiv Detail & Related papers (2021-05-05T08:38:25Z) - Conditional Convolutions for Instance Segmentation [109.2706837177222]
We propose a simple yet effective instance segmentation framework, termed CondInst.
We employ dynamic instance-aware networks, conditioned on instances.
We demonstrate a simpler instance segmentation method that can achieve improved performance in both accuracy and inference speed.
arXiv Detail & Related papers (2020-03-12T08:42:36Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.