Conditional Reconstruction for Open-set Semantic Segmentation

• URL: http://arxiv.org/abs/2203.01368v1
• Date: Wed, 2 Mar 2022 19:32:29 GMT
• Title: Conditional Reconstruction for Open-set Semantic Segmentation
• Authors: Ian Nunes, Matheus B. Pereira, Hugo Oliveira, Jefersson A. dos Santos, Marcus Poggi
• Abstract summary: We propose a novel method called CoReSeg thattackles the issue using class conditional reconstruction of the input images. It produces better se-mantic consistency in its predictions, resulting in cleanersegmentation maps. CoRe-Seg outperforms state-of-the-art methods on the Vaihin-gen and Potsdam ISPRS datasets.
• Score: 2.7771471571972333
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Open set segmentation is a relatively new and unexploredtask, with just a handful of methods proposed to model suchtasks.We propose a novel method called CoReSeg thattackles the issue using class conditional reconstruction ofthe input images according to their pixelwise mask. Ourmethod conditions each input pixel to all known classes,expecting higher errors for pixels of unknown classes. Itwas observed that the proposed method produces better se-mantic consistency in its predictions, resulting in cleanersegmentation maps that better fit object boundaries. CoRe-Seg outperforms state-of-the-art methods on the Vaihin-gen and Potsdam ISPRS datasets, while also being com-petitive on the Houston 2018 IEEE GRSS Data Fusiondataset. Official implementation for CoReSeg is availableat:https://github.com/iannunes/CoReSeg.

Related papers

• MixReorg: Cross-Modal Mixed Patch Reorganization is a Good Mask Learner for Open-World Semantic Segmentation [110.09800389100599]
  We propose MixReorg, a novel and straightforward pre-training paradigm for semantic segmentation. Our approach involves generating fine-grained patch-text pairs data by mixing image patches while preserving the correspondence between patches and text. With MixReorg as a mask learner, conventional text-supervised semantic segmentation models can achieve highly generalizable pixel-semantic alignment ability.
  arXiv  Detail & Related papers  (2023-08-09T09:35:16Z)
• CorrMatch: Label Propagation via Correlation Matching for Semi-Supervised Semantic Segmentation [73.89509052503222]
  This paper presents a simple but performant semi-supervised semantic segmentation approach, called CorrMatch. We observe that the correlation maps not only enable clustering pixels of the same category easily but also contain good shape information. We propose to conduct pixel propagation by modeling the pairwise similarities of pixels to spread the high-confidence pixels and dig out more. Then, we perform region propagation to enhance the pseudo labels with accurate class-agnostic masks extracted from the correlation maps.
  arXiv  Detail & Related papers  (2023-06-07T10:02:29Z)
• High-fidelity Pseudo-labels for Boosting Weakly-Supervised Segmentation [17.804090651425955]
  Image-level weakly-supervised segmentation (WSSS) reduces the usually vast data annotation cost by surrogate segmentation masks during training. Our work is based on two techniques for improving CAMs; importance sampling, which is a substitute for GAP, and the feature similarity loss. We reformulate both techniques based on binomial posteriors of multiple independent binary problems. This has two benefits; their performance is improved and they become more general, resulting in an add-on method that can boost virtually any WSSS method.
  arXiv  Detail & Related papers  (2023-04-05T17:43:57Z)
• ReFit: A Framework for Refinement of Weakly Supervised Semantic Segmentation using Object Border Fitting for Medical Images [4.945138408504987]
  Weakly Supervised Semantic (WSSS) relying only on image-level supervision is a promising approach to deal with the need for networks. We propose our novel ReFit framework, which deploys state-of-the-art class activation maps combined with various post-processing techniques. By applying our method to WSSS predictions, we achieved up to 10% improvement over the current state-of-the-art WSSS methods for medical imaging.
  arXiv  Detail & Related papers  (2023-03-14T12:46:52Z)
• FuSS: Fusing Superpixels for Improved Segmentation Consistency [2.7771471571972333]
  We propose two approaches to improve the semantic consistency of Open Set Semantic. First, we propose a method called OpenGMM to model the distribution of pixels for each class in a multimodal manner. The second approach is a post-processing which uses superpixels to enforce highly homogeneous regions to behave equally.
  arXiv  Detail & Related papers  (2022-06-06T16:14:19Z)
• RecurSeed and EdgePredictMix: Pseudo-Label Refinement Learning for Weakly Supervised Semantic Segmentation across Single- and Multi-Stage Frameworks [3.3240906432768482]
  weakly supervised semantic segmentation using only image-level labels (WSSS-IL) is potentially useful. But its low performance and implementation complexity still limit its application. We propose RecurSeed, which alternately reduces non- and false detections. We also propose a novel data augmentation (DA) approach called EdgePredictMix, which further expresses an object's edge.
  arXiv  Detail & Related papers  (2022-04-14T04:46:43Z)
• Open-World Instance Segmentation: Exploiting Pseudo Ground Truth From Learned Pairwise Affinity [59.1823948436411]
  We propose a novel approach for mask proposals, Generic Grouping Networks (GGNs) Our approach combines a local measure of pixel affinity with instance-level mask supervision, producing a training regimen designed to make the model as generic as the data diversity allows.
  arXiv  Detail & Related papers  (2022-04-12T22:37:49Z)
• Few-shot semantic segmentation via mask aggregation [5.886986014593717]
  Few-shot semantic segmentation aims to recognize novel classes with only very few labelled data. Previous works have typically regarded it as a pixel-wise classification problem. We introduce a mask-based classification method for addressing this problem.
  arXiv  Detail & Related papers  (2022-02-15T07:13:09Z)
• Half-Real Half-Fake Distillation for Class-Incremental Semantic Segmentation [84.1985497426083]
  convolutional neural networks are ill-equipped for incremental learning. New classes are available but the initial training data is not retained. We try to address this issue by "inverting" the trained segmentation network to synthesize input images starting from random noise.
  arXiv  Detail & Related papers  (2021-04-02T03:47:16Z)
• Exploring Cross-Image Pixel Contrast for Semantic Segmentation [130.22216825377618]
  We propose a pixel-wise contrastive framework for semantic segmentation in the fully supervised setting. The core idea is to enforce pixel embeddings belonging to a same semantic class to be more similar than embeddings from different classes. Our method can be effortlessly incorporated into existing segmentation frameworks without extra overhead during testing.
  arXiv  Detail & Related papers  (2021-01-28T11:35:32Z)
• Scaling Semantic Segmentation Beyond 1K Classes on a Single GPU [87.48110331544885]
  We propose a novel training methodology to train and scale the existing semantic segmentation models. We demonstrate a clear benefit of our approach on a dataset with 1284 classes, bootstrapped from LVIS and COCO annotations, with three times better mIoU than the DeeplabV3+ model.
  arXiv  Detail & Related papers  (2020-12-14T13:12:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.