Exposing Semantic Segmentation Failures via Maximum Discrepancy Competition

• URL: http://arxiv.org/abs/2103.00259v2
• Date: Wed, 3 Mar 2021 14:22:13 GMT
• Title: Exposing Semantic Segmentation Failures via Maximum Discrepancy Competition
• Authors: Jiebin Yan, Yu Zhong, Yuming Fang, Zhangyang Wang, Kede Ma
• Abstract summary: We take steps toward answering the question by exposing failures of existing semantic segmentation methods in the open visual world. Inspired by previous research on model falsification, we start from an arbitrarily large image set, and automatically sample a small image set by MAximizing the Discrepancy (MAD) between two segmentation methods. The selected images have the greatest potential in falsifying either (or both) of the two methods. A segmentation method, whose failures are more difficult to be exposed in the MAD competition, is considered better.
• Score: 102.75463782627791
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Semantic segmentation is an extensively studied task in computer vision, with numerous methods proposed every year. Thanks to the advent of deep learning in semantic segmentation, the performance on existing benchmarks is close to saturation. A natural question then arises: Does the superior performance on the closed (and frequently re-used) test sets transfer to the open visual world with unconstrained variations? In this paper, we take steps toward answering the question by exposing failures of existing semantic segmentation methods in the open visual world under the constraint of very limited human labeling effort. Inspired by previous research on model falsification, we start from an arbitrarily large image set, and automatically sample a small image set by MAximizing the Discrepancy (MAD) between two segmentation methods. The selected images have the greatest potential in falsifying either (or both) of the two methods. We also explicitly enforce several conditions to diversify the exposed failures, corresponding to different underlying root causes. A segmentation method, whose failures are more difficult to be exposed in the MAD competition, is considered better. We conduct a thorough MAD diagnosis of ten PASCAL VOC semantic segmentation algorithms. With detailed analysis of experimental results, we point out strengths and weaknesses of the competing algorithms, as well as potential research directions for further advancement in semantic segmentation. The codes are publicly available at \url{https://github.com/QTJiebin/MAD_Segmentation}.

Related papers

• Unsupervised Universal Image Segmentation [59.0383635597103]
  We propose an Unsupervised Universal model (U2Seg) adept at performing various image segmentation tasks. U2Seg generates pseudo semantic labels for these segmentation tasks via leveraging self-supervised models. We then self-train the model on these pseudo semantic labels, yielding substantial performance gains.
  arXiv  Detail & Related papers  (2023-12-28T18:59:04Z)
• SegRefiner: Towards Model-Agnostic Segmentation Refinement with Discrete Diffusion Process [102.18226145874007]
  We propose a model-agnostic solution called SegRefiner to enhance the quality of object masks produced by different segmentation models. SegRefiner takes coarse masks as inputs and refines them using a discrete diffusion process. It consistently improves both the segmentation metrics and boundary metrics across different types of coarse masks.
  arXiv  Detail & Related papers  (2023-12-19T18:53:47Z)
• FixMatchSeg: Fixing FixMatch for Semi-Supervised Semantic Segmentation [0.24366811507669117]
  Supervised deep learning methods for semantic medical image segmentation are getting increasingly popular in the past few years. In resource constrained settings, getting large number of annotated images is very difficult as it mostly requires experts. In this work, we adapt a state-of-the-art semi-supervised classification method FixMatch to semantic segmentation task, introducing FixMatchSeg.
  arXiv  Detail & Related papers  (2022-07-31T09:14:52Z)
• Hypernet-Ensemble Learning of Segmentation Probability for Medical Image Segmentation with Ambiguous Labels [8.841870931360585]
  Deep Learning approaches are notoriously overconfident about their prediction with highly polarized label probability. This is often not desirable for many applications with the inherent label ambiguity even in human annotations. We propose novel methods to improve the segmentation probability estimation without sacrificing performance in a real-world scenario.
  arXiv  Detail & Related papers  (2021-12-13T14:24:53Z)
• Unsupervised Semantic Segmentation by Contrasting Object Mask Proposals [78.12377360145078]
  We introduce a novel two-step framework that adopts a predetermined prior in a contrastive optimization objective to learn pixel embeddings. This marks a large deviation from existing works that relied on proxy tasks or end-to-end clustering. In particular, when fine-tuning the learned representations using just 1% of labeled examples on PASCAL, we outperform supervised ImageNet pre-training by 7.1% mIoU.
  arXiv  Detail & Related papers  (2021-02-11T18:54:47Z)
• Information-Theoretic Segmentation by Inpainting Error Maximization [30.520622129165456]
  We group image pixels into foreground and background, with the goal of minimizing predictability of one set from the other. Our method does not involve training deep networks, is computationally cheap, class-agnostic, and even applicable in isolation to a single unlabeled image.
  arXiv  Detail & Related papers  (2020-12-14T06:42:27Z)
• DEAL: Difficulty-aware Active Learning for Semantic Segmentation [33.96850316081623]
  Active learning aims to address the paucity of labeled data by finding the most informative samples. We propose a semantic Difficulty-awarE Active Learning network composed of two branches: the common segmentation branch and the semantic difficulty branch. For the latter branch, with the supervision of segmentation error between the segmentation result and GT, a pixel-wise probability attention module is introduced to learn the semantic difficulty scores for different semantic areas. Two acquisition functions are devised to select the most valuable samples with semantic difficulty.
  arXiv  Detail & Related papers  (2020-10-17T03:25:25Z)
• Contrastive Rendering for Ultrasound Image Segmentation [59.23915581079123]
  The lack of sharp boundaries in US images remains an inherent challenge for segmentation. We propose a novel and effective framework to improve boundary estimation in US images. Our proposed method outperforms state-of-the-art methods and has the potential to be used in clinical practice.
  arXiv  Detail & Related papers  (2020-10-10T07:14:03Z)
• Causal Intervention for Weakly-Supervised Semantic Segmentation [122.1846968696862]
  We aim to generate better pixel-level pseudo-masks by using only image-level labels. We propose a structural causal model to analyze the causalities among images, contexts, and class labels. Based on it, we develop a new method: Context Adjustment (CONTA), to remove the confounding bias in image-level classification.
  arXiv  Detail & Related papers  (2020-09-26T09:26:29Z)
• Human-Centered Unsupervised Segmentation Fusion [0.0]
  We introduce a new segmentation fusion model that is based on K-Modes clustering. Results obtained from publicly available datasets with human ground truth segmentations clearly show that our model outperforms the state-of-the-art on human segmentations.
  arXiv  Detail & Related papers  (2020-07-22T12:18:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.