Benchmarking the Robustness of Instance Segmentation Models

• URL: http://arxiv.org/abs/2109.01123v3
• Date: Fri, 28 Mar 2025 18:46:44 GMT
• Title: Benchmarking the Robustness of Instance Segmentation Models
• Authors: Yusuf Dalva, Hamza Pehlivan, Said Fahri Altindis, Aysegul Dundar,
• Abstract summary: This paper presents a comprehensive evaluation of instance segmentation models with respect to real-world image corruptions and out-of-domain image collections.<n>We find that group normalization enhances the robustness of networks across corruptions where the image contents stay the same but corruptions are added on top.<n>We also find that single-stage detectors do not generalize well to larger image resolutions than their training size.
• Score: 7.1699725781322465
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper presents a comprehensive evaluation of instance segmentation models with respect to real-world image corruptions as well as out-of-domain image collections, e.g. images captured by a different set-up than the training dataset. The out-of-domain image evaluation shows the generalization capability of models, an essential aspect of real-world applications and an extensively studied topic of domain adaptation. These presented robustness and generalization evaluations are important when designing instance segmentation models for real-world applications and picking an off-the-shelf pretrained model to directly use for the task at hand. Specifically, this benchmark study includes state-of-the-art network architectures, network backbones, normalization layers, models trained starting from scratch versus pretrained networks, and the effect of multi-task training on robustness and generalization. Through this study, we gain several insights. For example, we find that group normalization enhances the robustness of networks across corruptions where the image contents stay the same but corruptions are added on top. On the other hand, batch normalization improves the generalization of the models across different datasets where statistics of image features change. We also find that single-stage detectors do not generalize well to larger image resolutions than their training size. On the other hand, multi-stage detectors can easily be used on images of different sizes. We hope that our comprehensive study will motivate the development of more robust and reliable instance segmentation models.

Related papers

• Low-Biased General Annotated Dataset Generation [62.04202037186855]
  We present a low-biased general annotated dataset generation framework (lbGen) Instead of expensive manual collection, we aim at directly generating low-biased images with category annotations. Experimental results confirm that, compared with the manually labeled dataset or other synthetic datasets, the utilization of our generated low-biased dataset leads to stable generalization capacity enhancement.
  arXiv  Detail & Related papers  (2024-12-14T13:28:40Z)
• FoundIR: Unleashing Million-scale Training Data to Advance Foundation Models for Image Restoration [66.61201445650323]
  Existing methods suffer from a generalization bottleneck in real-world scenarios. We contribute a million-scale dataset with two notable advantages over existing training data. We propose a robust model, FoundIR, to better address a broader range of restoration tasks in real-world scenarios.
  arXiv  Detail & Related papers  (2024-12-02T12:08:40Z)
• In Search of Forgotten Domain Generalization [20.26519807919284]
  Out-of-Domain (OOD) generalization is the ability of a model trained on one or more domains to generalize to unseen domains. In the ImageNet era of computer vision, evaluation sets for measuring a model's OOD performance were designed to be strictly OOD with respect to style. The emergence of foundation models and expansive web-scale datasets has obfuscated this evaluation process.
  arXiv  Detail & Related papers  (2024-10-10T17:50:45Z)
• Zero-Shot Object-Centric Representation Learning [72.43369950684057]
  We study current object-centric methods through the lens of zero-shot generalization. We introduce a benchmark comprising eight different synthetic and real-world datasets. We find that training on diverse real-world images improves transferability to unseen scenarios.
  arXiv  Detail & Related papers  (2024-08-17T10:37:07Z)
• GM-DF: Generalized Multi-Scenario Deepfake Detection [49.072106087564144]
  Existing face forgery detection usually follows the paradigm of training models in a single domain. In this paper, we elaborately investigate the generalization capacity of deepfake detection models when jointly trained on multiple face forgery detection datasets.
  arXiv  Detail & Related papers  (2024-06-28T17:42:08Z)
• Reinforcing Pre-trained Models Using Counterfactual Images [54.26310919385808]
  This paper proposes a novel framework to reinforce classification models using language-guided generated counterfactual images. We identify model weaknesses by testing the model using the counterfactual image dataset. We employ the counterfactual images as an augmented dataset to fine-tune and reinforce the classification model.
  arXiv  Detail & Related papers  (2024-06-19T08:07:14Z)
• MTP: Advancing Remote Sensing Foundation Model via Multi-Task Pretraining [73.81862342673894]
  Foundation models have reshaped the landscape of Remote Sensing (RS) by enhancing various image interpretation tasks. transferring the pretrained models to downstream tasks may encounter task discrepancy due to their formulation of pretraining as image classification or object discrimination tasks. We conduct multi-task supervised pretraining on the SAMRS dataset, encompassing semantic segmentation, instance segmentation, and rotated object detection. Our models are finetuned on various RS downstream tasks, such as scene classification, horizontal and rotated object detection, semantic segmentation, and change detection.
  arXiv  Detail & Related papers  (2024-03-20T09:17:22Z)
• Benchmarking Zero-Shot Robustness of Multimodal Foundation Models: A Pilot Study [61.65123150513683]
  multimodal foundation models, such as CLIP, produce state-of-the-art zero-shot results. It is reported that these models close the robustness gap by matching the performance of supervised models trained on ImageNet. We show that CLIP leads to a significant robustness drop compared to supervised ImageNet models on our benchmark.
  arXiv  Detail & Related papers  (2024-03-15T17:33:49Z)
• Generalization Properties of Retrieval-based Models [50.35325326050263]
  Retrieval-based machine learning methods have enjoyed success on a wide range of problems. Despite growing literature showcasing the promise of these models, the theoretical underpinning for such models remains underexplored. We present a formal treatment of retrieval-based models to characterize their generalization ability.
  arXiv  Detail & Related papers  (2022-10-06T00:33:01Z)
• More is Better: A Novel Multi-view Framework for Domain Generalization [28.12350681444117]
  Key issue of domain generalization (DG) is how to prevent overfitting to the observed source domains. By treating tasks and images as different views, we propose a novel multi-view DG framework. In test stage, to alleviate unstable prediction, we utilize multiple augmented images to yield multi-view prediction.
  arXiv  Detail & Related papers  (2021-12-23T02:51:35Z)
• Rethinking Self-Supervision Objectives for Generalizable Coherence Modeling [8.329870357145927]
  Coherence evaluation of machine generated text is one of the principal applications of coherence models that needs to be investigated. We explore training data and self-supervision objectives that result in a model that generalizes well across tasks. We show empirically that increasing the density of negative samples improves the basic model, and using a global negative queue further improves and stabilizes the model while training with hard negative samples.
  arXiv  Detail & Related papers  (2021-10-14T07:44:14Z)
• Salient Objects in Clutter [130.63976772770368]
  This paper identifies and addresses a serious design bias of existing salient object detection (SOD) datasets. This design bias has led to a saturation in performance for state-of-the-art SOD models when evaluated on existing datasets. We propose a new high-quality dataset and update the previous saliency benchmark.
  arXiv  Detail & Related papers  (2021-05-07T03:49:26Z)
• Generative Interventions for Causal Learning [27.371436971655303]
  We introduce a framework for learning robust visual representations that generalize to new viewpoints, backgrounds, and scene contexts. We show that we can steer generative models to manufacture interventions on features caused by confounding factors.
  arXiv  Detail & Related papers  (2020-12-22T16:01:55Z)
• Multi-task pre-training of deep neural networks for digital pathology [8.74883469030132]
  We first assemble and transform many digital pathology datasets into a pool of 22 classification tasks and almost 900k images. We show that our models used as feature extractors either improve significantly over ImageNet pre-trained models or provide comparable performance.
  arXiv  Detail & Related papers  (2020-05-05T08:50:17Z)
• Generalizable Model-agnostic Semantic Segmentation via Target-specific Normalization [24.14272032117714]
  We propose a novel domain generalization framework for the generalizable semantic segmentation task. We exploit the model-agnostic learning to simulate the domain shift problem. Considering the data-distribution discrepancy between seen source and unseen target domains, we develop the target-specific normalization scheme.
  arXiv  Detail & Related papers  (2020-03-27T09:25:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.