Tree Energy Loss: Towards Sparsely Annotated Semantic Segmentation
- URL: http://arxiv.org/abs/2203.10739v2
- Date: Tue, 22 Mar 2022 04:17:36 GMT
- Title: Tree Energy Loss: Towards Sparsely Annotated Semantic Segmentation
- Authors: Zhiyuan Liang, Tiancai Wang, Xiangyu Zhang, Jian Sun, Jianbing Shen
- Abstract summary: Sparsely annotated semantic segmentation (SASS) aims to train a segmentation network with coarse-grained supervisions.
We propose a novel tree energy loss for SASS by providing semantic guidance for unlabeled pixels.
- Score: 141.16965264264195
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Sparsely annotated semantic segmentation (SASS) aims to train a segmentation
network with coarse-grained (i.e., point-, scribble-, and block-wise)
supervisions, where only a small proportion of pixels are labeled in each
image. In this paper, we propose a novel tree energy loss for SASS by providing
semantic guidance for unlabeled pixels. The tree energy loss represents images
as minimum spanning trees to model both low-level and high-level pair-wise
affinities. By sequentially applying these affinities to the network
prediction, soft pseudo labels for unlabeled pixels are generated in a
coarse-to-fine manner, achieving dynamic online self-training. The tree energy
loss is effective and easy to be incorporated into existing frameworks by
combining it with a traditional segmentation loss. Compared with previous SASS
methods, our method requires no multistage training strategies, alternating
optimization procedures, additional supervised data, or time-consuming
post-processing while outperforming them in all SASS settings. Code is
available at https://github.com/megvii-research/TreeEnergyLoss.
Related papers
- Weakly Supervised LiDAR Semantic Segmentation via Scatter Image Annotation [38.715754110667916]
We implement LiDAR semantic segmentation using scatter image annotation.
We also propose ScatterNet, a network that includes three pivotal strategies to reduce the performance gap.
Our method requires less than 0.02% of the labeled points to achieve over 95% of the performance of fully-supervised methods.
arXiv Detail & Related papers (2024-04-19T13:01:30Z) - Weakly-supervised fire segmentation by visualizing intermediate CNN
layers [82.75113406937194]
Fire localization in images and videos is an important step for an autonomous system to combat fire incidents.
We consider weakly supervised segmentation of fire in images, in which only image labels are used to train the network.
We show that in the case of fire segmentation, which is a binary segmentation problem, the mean value of features in a mid-layer of classification CNN can perform better than conventional Class Activation Mapping (CAM) method.
arXiv Detail & Related papers (2021-11-16T11:56:28Z) - Maximize the Exploration of Congeneric Semantics for Weakly Supervised
Semantic Segmentation [27.155133686127474]
We construct a graph neural network (P-GNN) based on the self-detected patches from different images that contain the same class labels.
We conduct experiments on the popular PASCAL VOC 2012 benchmarks, and our model yields state-of-the-art performance.
arXiv Detail & Related papers (2021-10-08T08:59:16Z) - Semantic Segmentation with Generative Models: Semi-Supervised Learning
and Strong Out-of-Domain Generalization [112.68171734288237]
We propose a novel framework for discriminative pixel-level tasks using a generative model of both images and labels.
We learn a generative adversarial network that captures the joint image-label distribution and is trained efficiently using a large set of unlabeled images.
We demonstrate strong in-domain performance compared to several baselines, and are the first to showcase extreme out-of-domain generalization.
arXiv Detail & Related papers (2021-04-12T21:41:25Z) - Weakly-Supervised Image Semantic Segmentation Using Graph Convolutional
Networks [9.066817971329899]
weakly-supervised image semantic segmentation based on image-level class labels.
One common approach to this task is to propagate the activation scores of Class Activation Maps (CAMs) using a random-walk mechanism.
We propose a Graph Convolutional Network (GCN)-based feature propagation framework.
arXiv Detail & Related papers (2021-03-31T02:05:01Z) - Group-Wise Semantic Mining for Weakly Supervised Semantic Segmentation [49.90178055521207]
This work addresses weakly supervised semantic segmentation (WSSS), with the goal of bridging the gap between image-level annotations and pixel-level segmentation.
We formulate WSSS as a novel group-wise learning task that explicitly models semantic dependencies in a group of images to estimate more reliable pseudo ground-truths.
In particular, we devise a graph neural network (GNN) for group-wise semantic mining, wherein input images are represented as graph nodes.
arXiv Detail & Related papers (2020-12-09T12:40:13Z) - Sequential Graph Convolutional Network for Active Learning [53.99104862192055]
We propose a novel pool-based Active Learning framework constructed on a sequential Graph Convolution Network (GCN)
With a small number of randomly sampled images as seed labelled examples, we learn the parameters of the graph to distinguish labelled vs unlabelled nodes.
We exploit these characteristics of GCN to select the unlabelled examples which are sufficiently different from labelled ones.
arXiv Detail & Related papers (2020-06-18T00:55:10Z) - Weakly-Supervised Semantic Segmentation by Iterative Affinity Learning [86.45526827323954]
Weakly-supervised semantic segmentation is a challenging task as no pixel-wise label information is provided for training.
We propose an iterative algorithm to learn such pairwise relations.
We show that the proposed algorithm performs favorably against the state-of-the-art methods.
arXiv Detail & Related papers (2020-02-19T10:32:03Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.