Rethinking Semantic Segmentation Evaluation for Explainability and Model Selection

• URL: http://arxiv.org/abs/2101.08418v1
• Date: Thu, 21 Jan 2021 03:12:43 GMT
• Title: Rethinking Semantic Segmentation Evaluation for Explainability and Model Selection
• Authors: Yuxiang Zhang, Sachin Mehta, Anat Caspi
• Abstract summary: We introduce a new metric to assess region-based over- and under-segmentation. We analyze and compare it to other metrics, demonstrating that the use of our metric lends greater explainability to semantic segmentation model performance in real-world applications.
• Score: 12.786648212233116
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Semantic segmentation aims to robustly predict coherent class labels for entire regions of an image. It is a scene understanding task that powers real-world applications (e.g., autonomous navigation). One important application, the use of imagery for automated semantic understanding of pedestrian environments, provides remote mapping of accessibility features in street environments. This application (and others like it) require detailed geometric information of geographical objects. Semantic segmentation is a prerequisite for this task since it maps contiguous regions of the same class as single entities. Importantly, semantic segmentation uses like ours are not pixel-wise outcomes; however, most of their quantitative evaluation metrics (e.g., mean Intersection Over Union) are based on pixel-wise similarities to a ground-truth, which fails to emphasize over- and under-segmentation properties of a segmentation model. Here, we introduce a new metric to assess region-based over- and under-segmentation. We analyze and compare it to other metrics, demonstrating that the use of our metric lends greater explainability to semantic segmentation model performance in real-world applications.

Related papers

• RoboHop: Segment-based Topological Map Representation for Open-World Visual Navigation [18.053914853235142]
  We propose a novel representation of an environment based on "image segments" Unlike 3D scene graphs, we create a purely topological graph with segments as nodes. This unveils a "continuous sense of a place", defined by inter-image persistence of segments.
  arXiv  Detail & Related papers  (2024-05-09T14:17:26Z)
• Semantic Segmentation by Semantic Proportions [0.0]
  We propose a novel approach for semantic segmentation by eliminating the need of ground-truth segmentation maps. Our approach requires only the rough information of individual semantic class proportions, shortened as semantic proportions. It greatly simplifies the data annotation process and thus will significantly reduce the annotation time and cost.
  arXiv  Detail & Related papers  (2023-05-24T22:51:52Z)
• Unsupervised Domain Adaptation for Medical Image Segmentation via Feature-space Density Matching [0.0]
  This paper presents an unsupervised domain adaptation approach for semantic segmentation. We match the target data distribution to the source in the feature space, particularly when the number of target samples is limited. We demonstrate the efficacy of our proposed approach on 2 datasets, multisite prostate MRI and histopathology images.
  arXiv  Detail & Related papers  (2023-05-09T22:24:46Z)
• Open-world Semantic Segmentation via Contrasting and Clustering Vision-Language Embedding [95.78002228538841]
  We propose a new open-world semantic segmentation pipeline that makes the first attempt to learn to segment semantic objects of various open-world categories without any efforts on dense annotations. Our method can directly segment objects of arbitrary categories, outperforming zero-shot segmentation methods that require data labeling on three benchmark datasets.
  arXiv  Detail & Related papers  (2022-07-18T09:20:04Z)
• SemAffiNet: Semantic-Affine Transformation for Point Cloud Segmentation [94.11915008006483]
  We propose SemAffiNet for point cloud semantic segmentation. We conduct extensive experiments on the ScanNetV2 and NYUv2 datasets.
  arXiv  Detail & Related papers  (2022-05-26T17:00:23Z)
• Scaling up Multi-domain Semantic Segmentation with Sentence Embeddings [81.09026586111811]
  We propose an approach to semantic segmentation that achieves state-of-the-art supervised performance when applied in a zero-shot setting. This is achieved by replacing each class label with a vector-valued embedding of a short paragraph that describes the class. The resulting merged semantic segmentation dataset of over 2 Million images enables training a model that achieves performance equal to that of state-of-the-art supervised methods on 7 benchmark datasets.
  arXiv  Detail & Related papers  (2022-02-04T07:19:09Z)
• TransFGU: A Top-down Approach to Fine-Grained Unsupervised Semantic Segmentation [44.75300205362518]
  Unsupervised semantic segmentation aims to obtain high-level semantic representation on low-level visual features without manual annotations. We propose the first top-down unsupervised semantic segmentation framework for fine-grained segmentation in extremely complicated scenarios. Our results show that our top-down unsupervised segmentation is robust to both object-centric and scene-centric datasets.
  arXiv  Detail & Related papers  (2021-12-02T18:59:03Z)
• Robust 3D Scene Segmentation through Hierarchical and Learnable Part-Fusion [9.275156524109438]
  3D semantic segmentation is a fundamental building block for several scene understanding applications such as autonomous driving, robotics and AR/VR. Previous methods have utilized hierarchical, iterative methods to fuse semantic and instance information, but they lack learnability in context fusion. This paper presents Segment-Fusion, a novel attention-based method for hierarchical fusion of semantic and instance information.
  arXiv  Detail & Related papers  (2021-11-16T13:14:47Z)
• SegmentMeIfYouCan: A Benchmark for Anomaly Segmentation [111.61261419566908]
  Deep neural networks (DNNs) are usually trained on a closed set of semantic classes. They are ill-equipped to handle previously-unseen objects. detecting and localizing such objects is crucial for safety-critical applications such as perception for automated driving.
  arXiv  Detail & Related papers  (2021-04-30T07:58:19Z)
• Mining Cross-Image Semantics for Weakly Supervised Semantic Segmentation [128.03739769844736]
  Two neural co-attentions are incorporated into the classifier to capture cross-image semantic similarities and differences. In addition to boosting object pattern learning, the co-attention can leverage context from other related images to improve localization map inference. Our algorithm sets new state-of-the-arts on all these settings, demonstrating well its efficacy and generalizability.
  arXiv  Detail & Related papers  (2020-07-03T21:53:46Z)
• SceneEncoder: Scene-Aware Semantic Segmentation of Point Clouds with A Learnable Scene Descriptor [51.298760338410624]
  We propose a SceneEncoder module to impose a scene-aware guidance to enhance the effect of global information. The module predicts a scene descriptor, which learns to represent the categories of objects existing in the scene. We also design a region similarity loss to propagate distinguishing features to their own neighboring points with the same label.
  arXiv  Detail & Related papers  (2020-01-24T16:53:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.