SegmentMeIfYouCan: A Benchmark for Anomaly Segmentation

• URL: http://arxiv.org/abs/2104.14812v1
• Date: Fri, 30 Apr 2021 07:58:19 GMT
• Title: SegmentMeIfYouCan: A Benchmark for Anomaly Segmentation
• Authors: Robin Chan, Krzysztof Lis, Svenja Uhlemeyer, Hermann Blum, Sina Honari, Roland Siegwart, Mathieu Salzmann, Pascal Fua and Matthias Rottmann
• Abstract summary: Deep neural networks (DNNs) are usually trained on a closed set of semantic classes. They are ill-equipped to handle previously-unseen objects. detecting and localizing such objects is crucial for safety-critical applications such as perception for automated driving.
• Score: 111.61261419566908
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: State-of-the-art semantic or instance segmentation deep neural networks (DNNs) are usually trained on a closed set of semantic classes. As such, they are ill-equipped to handle previously-unseen objects. However, detecting and localizing such objects is crucial for safety-critical applications such as perception for automated driving, especially if they appear on the road ahead. While some methods have tackled the tasks of anomalous or out-of-distribution object segmentation, progress remains slow, in large part due to the lack of solid benchmarks; existing datasets either consist of synthetic data, or suffer from label inconsistencies. In this paper, we bridge this gap by introducing the "SegmentMeIfYouCan" benchmark. Our benchmark addresses two tasks: Anomalous object segmentation, which considers any previously-unseen object category; and road obstacle segmentation, which focuses on any object on the road, may it be known or unknown. We provide two corresponding datasets together with a test suite performing an in-depth method analysis, considering both established pixel-wise performance metrics and recent component-wise ones, which are insensitive to object sizes. We empirically evaluate multiple state-of-the-art baseline methods, including several specifically designed for anomaly / obstacle segmentation, on our datasets as well as on public ones, using our benchmark suite. The anomaly and obstacle segmentation results show that our datasets contribute to the diversity and challengingness of both dataset landscapes.

Related papers

• Segmenting Object Affordances: Reproducibility and Sensitivity to Scale [27.277739855754447]
  Methods re-use and adapt learning-based architectures for semantic segmentation to the affordance segmentation task. We benchmark these methods under a reproducible setup on two single objects scenarios. Our analysis shows that models are not robust to scale variations when object resolutions differ from those in the training set.
  arXiv  Detail & Related papers  (2024-09-03T11:54:36Z)
• OoDIS: Anomaly Instance Segmentation Benchmark [57.89836988990543]
  We extend the most commonly used anomaly segmentation benchmarks to include the instance segmentation task. Development in this area has been lagging, largely due to the lack of dedicated benchmarks. Our evaluation of anomaly instance segmentation methods shows that this challenge remains an unsolved problem.
  arXiv  Detail & Related papers  (2024-06-17T17:59:56Z)
• Frequency-based Matcher for Long-tailed Semantic Segmentation [22.199174076366003]
  We focus on a relatively under-explored task setting, long-tailed semantic segmentation (LTSS) We propose a dual-metric evaluation system and construct the LTSS benchmark to demonstrate the performance of semantic segmentation methods and long-tailed solutions. We also propose a transformer-based algorithm to improve LTSS, frequency-based matcher, which solves the oversuppression problem by one-to-many matching.
  arXiv  Detail & Related papers  (2024-06-06T09:57:56Z)
• ISAR: A Benchmark for Single- and Few-Shot Object Instance Segmentation and Re-Identification [24.709695178222862]
  We propose ISAR, a benchmark and baseline method for single- and few-shot object identification. We provide a semi-synthetic dataset of video sequences with ground-truth semantic annotations. Our benchmark aligns with the emerging research trend of unifying Multi-Object Tracking, Video Object, and Re-identification.
  arXiv  Detail & Related papers  (2023-11-05T18:51:33Z)
• BURST: A Benchmark for Unifying Object Recognition, Segmentation and Tracking in Video [58.71785546245467]
  Multiple existing benchmarks involve tracking and segmenting objects in video. There is little interaction between them due to the use of disparate benchmark datasets and metrics. We propose BURST, a dataset which contains thousands of diverse videos with high-quality object masks. All tasks are evaluated using the same data and comparable metrics, which enables researchers to consider them in unison.
  arXiv  Detail & Related papers  (2022-09-25T01:27:35Z)
• Scaling up Multi-domain Semantic Segmentation with Sentence Embeddings [81.09026586111811]
  We propose an approach to semantic segmentation that achieves state-of-the-art supervised performance when applied in a zero-shot setting. This is achieved by replacing each class label with a vector-valued embedding of a short paragraph that describes the class. The resulting merged semantic segmentation dataset of over 2 Million images enables training a model that achieves performance equal to that of state-of-the-art supervised methods on 7 benchmark datasets.
  arXiv  Detail & Related papers  (2022-02-04T07:19:09Z)
• Target-Aware Object Discovery and Association for Unsupervised Video Multi-Object Segmentation [79.6596425920849]
  This paper addresses the task of unsupervised video multi-object segmentation. We introduce a novel approach for more accurate and efficient unseen-temporal segmentation. We evaluate the proposed approach on DAVIS$_17$ and YouTube-VIS, and the results demonstrate that it outperforms state-of-the-art methods both in segmentation accuracy and inference speed.
  arXiv  Detail & Related papers  (2021-04-10T14:39:44Z)
• Video Class Agnostic Segmentation Benchmark for Autonomous Driving [13.312978643938202]
  In certain safety-critical robotics applications, it is important to segment all objects, including those unknown at training time. We formalize the task of video class segmentation from monocular video sequences in autonomous driving to account for unknown objects.
  arXiv  Detail & Related papers  (2021-03-19T20:41:40Z)
• PointFlow: Flowing Semantics Through Points for Aerial Image Segmentation [96.76882806139251]
  We propose a point-wise affinity propagation module based on the Feature Pyramid Network (FPN) framework, named PointFlow. Rather than dense affinity learning, a sparse affinity map is generated upon selected points between the adjacent features. Experimental results on three different aerial segmentation datasets suggest that the proposed method is more effective and efficient than state-of-the-art general semantic segmentation methods.
  arXiv  Detail & Related papers  (2021-03-11T09:42:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.