BoxTeacher: Exploring High-Quality Pseudo Labels for Weakly Supervised
Instance Segmentation
- URL: http://arxiv.org/abs/2210.05174v1
- Date: Tue, 11 Oct 2022 06:23:30 GMT
- Title: BoxTeacher: Exploring High-Quality Pseudo Labels for Weakly Supervised
Instance Segmentation
- Authors: Tianheng Cheng and Xinggang Wang and Shaoyu Chen and Qian Zhang and
Wenyu Liu
- Abstract summary: BoxTeacher is an efficient and end-to-end training framework for high-performance weakly supervised instance segmentation.
We present a mask-aware confidence score to estimate the quality of pseudo masks, and propose the noise-aware pixel loss and noise-reduced affinity loss to adaptively optimize the student with pseudo masks.
- Score: 33.64088504387974
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Labeling objects with pixel-wise segmentation requires a huge amount of human
labor compared to bounding boxes. Most existing methods for weakly supervised
instance segmentation focus on designing heuristic losses with priors from
bounding boxes. While, we find that box-supervised methods can produce some
fine segmentation masks and we wonder whether the detectors could learn from
these fine masks while ignoring low-quality masks. To answer this question, we
present BoxTeacher, an efficient and end-to-end training framework for
high-performance weakly supervised instance segmentation, which leverages a
sophisticated teacher to generate high-quality masks as pseudo labels.
Considering the massive noisy masks hurt the training, we present a mask-aware
confidence score to estimate the quality of pseudo masks, and propose the
noise-aware pixel loss and noise-reduced affinity loss to adaptively optimize
the student with pseudo masks. Extensive experiments can demonstrate
effectiveness of the proposed BoxTeacher. Without bells and whistles,
BoxTeacher remarkably achieves $34.4$ mask AP and $35.4$ mask AP with ResNet-50
and ResNet-101 respectively on the challenging MS-COCO dataset, which
outperforms the previous state-of-the-art methods by a significant margin. The
code and models are available at \url{https://github.com/hustvl/BoxTeacher}.
Related papers
- Downstream Task Guided Masking Learning in Masked Autoencoders Using
Multi-Level Optimization [42.82742477950748]
Masked Autoencoder (MAE) is a notable method for self-supervised pretraining in visual representation learning.
We introduce the Multi-level Optimized Mask Autoencoder (MLO-MAE), a novel framework that learns an optimal masking strategy during pretraining.
Our experimental findings highlight MLO-MAE's significant advancements in visual representation learning.
arXiv Detail & Related papers (2024-02-28T07:37:26Z) - What You See is What You Classify: Black Box Attributions [61.998683569022006]
We train a deep network, the Explainer, to predict attributions for a pre-trained black-box classifier, the Explanandum.
Unlike most existing approaches, ours is capable of directly generating very distinct class-specific masks.
We show that our attributions are superior to established methods both visually and quantitatively.
arXiv Detail & Related papers (2022-05-23T12:30:04Z) - Mask Transfiner for High-Quality Instance Segmentation [95.74244714914052]
We present Mask Transfiner for high-quality and efficient instance segmentation.
Our approach only processes detected error-prone tree nodes and self-corrects their errors in parallel.
Our code and trained models will be available at http://vis.xyz/pub/transfiner.
arXiv Detail & Related papers (2021-11-26T18:58:22Z) - Open-Vocabulary Instance Segmentation via Robust Cross-Modal
Pseudo-Labeling [61.03262873980619]
Open-vocabulary instance segmentation aims at segmenting novel classes without mask annotations.
We propose a cross-modal pseudo-labeling framework, which generates training pseudo masks by aligning word semantics in captions with visual features of object masks in images.
Our framework is capable of labeling novel classes in captions via their word semantics to self-train a student model.
arXiv Detail & Related papers (2021-11-24T18:50:47Z) - Mask is All You Need: Rethinking Mask R-CNN for Dense and
Arbitrary-Shaped Scene Text Detection [11.390163890611246]
Mask R-CNN is widely adopted as a strong baseline for arbitrary-shaped scene text detection and spotting.
There may exist multiple instances in one proposal, which makes it difficult for the mask head to distinguish different instances and degrades the performance.
We propose instance-aware mask learning in which the mask head learns to predict the shape of the whole instance rather than classify each pixel to text or non-text.
arXiv Detail & Related papers (2021-09-08T04:32:29Z) - Box-Adapt: Domain-Adaptive Medical Image Segmentation using Bounding
BoxSupervision [52.45336255472669]
We propose a weakly supervised do-main adaptation setting for deep learning.
Box-Adapt fully explores the fine-grained segmenta-tion mask in the source domain and the weak bounding box in the target domain.
We demonstrate the effectiveness of our method in the liver segmentation task.
arXiv Detail & Related papers (2021-08-19T01:51:04Z) - Learning Class-Agnostic Pseudo Mask Generation for Box-Supervised
Semantic Segmentation [156.9155100983315]
We seek for a more accurate learning-based class-agnostic pseudo mask generator tailored to box-supervised semantic segmentation.
Our method can further close the performance gap between box-supervised and fully-supervised models.
arXiv Detail & Related papers (2021-03-09T14:54:54Z) - BoxInst: High-Performance Instance Segmentation with Box Annotations [102.10713189544947]
We present a high-performance method that can achieve mask-level instance segmentation with only bounding-box annotations for training.
Our core idea is to exploit the loss of learning masks in instance segmentation, with no modification to the segmentation network itself.
arXiv Detail & Related papers (2020-12-03T22:27:55Z) - Prior to Segment: Foreground Cues for Weakly Annotated Classes in
Partially Supervised Instance Segmentation [3.192503074844774]
Partially supervised instance segmentation aims to improve mask prediction with limited mask labels by utilizing the more abundant weak box labels.
We show that a class agnostic mask head, commonly used in partially supervised instance segmentation, has difficulties learning a general concept of foreground for the weakly annotated classes.
We introduce an object mask prior (OMP) that provides the mask head with the general concept of foreground implicitly learned by the box classification head under the supervision of all classes.
arXiv Detail & Related papers (2020-11-23T23:15:06Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.