Consensus Synergizes with Memory: A Simple Approach for Anomaly Segmentation in Urban Scenes

• URL: http://arxiv.org/abs/2111.15463v1
• Date: Wed, 24 Nov 2021 10:01:20 GMT
• Title: Consensus Synergizes with Memory: A Simple Approach for Anomaly Segmentation in Urban Scenes
• Authors: Jiazhong Cen, Zenkun Jiang, Lingxi Xie, Qi Tian, Xiaokang Yang, Wei Shen
• Abstract summary: Anomaly segmentation is a crucial task for safety-critical applications, such as autonomous driving in urban scenes. We propose a novel and simple approach named Consensus Synergizes with Memory (CosMe) to address this challenge. Experimental results on several urban scene anomaly segmentation datasets show that CosMe outperforms previous approaches by large margins.
• Score: 132.16748656557013
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Anomaly segmentation is a crucial task for safety-critical applications, such as autonomous driving in urban scenes, where the goal is to detect out-of-distribution (OOD) objects with categories which are unseen during training. The core challenge of this task is how to distinguish hard in-distribution samples from OOD samples, which has not been explicitly discussed yet. In this paper, we propose a novel and simple approach named Consensus Synergizes with Memory (CosMe) to address this challenge, inspired by the psychology finding that groups perform better than individuals on memory tasks. The main idea is 1) building a memory bank which consists of seen prototypes extracted from multiple layers of the pre-trained segmentation model and 2) training an auxiliary model that mimics the behavior of the pre-trained model, and then measuring the consensus of their mid-level features as complementary cues that synergize with the memory bank. CosMe is good at distinguishing between hard in-distribution examples and OOD samples. Experimental results on several urban scene anomaly segmentation datasets show that CosMe outperforms previous approaches by large margins.

Related papers

• NubbleDrop: A Simple Way to Improve Matching Strategy for Prompted One-Shot Segmentation [2.2559617939136505]
  We propose a simple and training-free method to enhance the validity and robustness of the matching strategy. The core concept involves randomly dropping feature channels (setting them to zero) during the matching process. This technique mimics discarding pathological nubbles, and it can be seamlessly applied to other similarity computing scenarios.
  arXiv  Detail & Related papers  (2024-05-19T08:00:38Z)
• Semantic-SAM: Segment and Recognize Anything at Any Granularity [83.64686655044765]
  We introduce Semantic-SAM, a universal image segmentation model to enable segment and recognize anything at any desired granularity. We consolidate multiple datasets across three granularities and introduce decoupled classification for objects and parts. For the multi-granularity capability, we propose a multi-choice learning scheme during training, enabling each click to generate masks at multiple levels.
  arXiv  Detail & Related papers  (2023-07-10T17:59:40Z)
• Weakly-Supervised Concealed Object Segmentation with SAM-based Pseudo Labeling and Multi-scale Feature Grouping [40.07070188661184]
  Weakly-Supervised Concealed Object (WSCOS) aims to segment objects well blended with surrounding environments. It is hard to distinguish concealed objects from the background due to the intrinsic similarity. We propose a new WSCOS method to address these two challenges.
  arXiv  Detail & Related papers  (2023-05-18T14:31:34Z)
• Revisiting the Evaluation of Image Synthesis with GANs [55.72247435112475]
  This study presents an empirical investigation into the evaluation of synthesis performance, with generative adversarial networks (GANs) as a representative of generative models. In particular, we make in-depth analyses of various factors, including how to represent a data point in the representation space, how to calculate a fair distance using selected samples, and how many instances to use from each set.
  arXiv  Detail & Related papers  (2023-04-04T17:54:32Z)
• Instance-Level Relative Saliency Ranking with Graph Reasoning [126.09138829920627]
  We present a novel unified model to segment salient instances and infer relative saliency rank order. A novel loss function is also proposed to effectively train the saliency ranking branch. experimental results demonstrate that our proposed model is more effective than previous methods.
  arXiv  Detail & Related papers  (2021-07-08T13:10:42Z)
• Zero-Shot Instance Segmentation [4.457471295379149]
  We propose a new task set named zero-shot instance segmentation (ZSI) In the training phase, the model is trained with seen data, while in the testing phase, it is used to segment all seen and unseen instances. We present a new benchmark for zero-shot instance segmentation based on the MS-COCO dataset.
  arXiv  Detail & Related papers  (2021-04-14T03:02:48Z)
• The Devil is in Classification: A Simple Framework for Long-tail Object Detection and Instance Segmentation [93.17367076148348]
  We investigate performance drop of the state-of-the-art two-stage instance segmentation model Mask R-CNN on the recent long-tail LVIS dataset. We unveil that a major cause is the inaccurate classification of object proposals. We propose a simple calibration framework to more effectively alleviate classification head bias with a bi-level class balanced sampling approach.
  arXiv  Detail & Related papers  (2020-07-23T12:49:07Z)
• UniT: Unified Knowledge Transfer for Any-shot Object Detection and Segmentation [52.487469544343305]
  Methods for object detection and segmentation rely on large scale instance-level annotations for training. We propose an intuitive and unified semi-supervised model that is applicable to a range of supervision.
  arXiv  Detail & Related papers  (2020-06-12T22:45:47Z)
• Dynamic Sampling for Deep Metric Learning [7.010669841466896]
  Deep metric learning maps visually similar images onto nearby locations and visually dissimilar images apart from each other in an embedding manifold. A dynamic sampling strategy is proposed to organize the training pairs in an easy-to-hard order to feed into the network. It allows the network to learn general boundaries between categories from the easy training pairs at its early stages and finalize the details of the model mainly relying on the hard training samples in the later.
  arXiv  Detail & Related papers  (2020-04-24T09:47:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.