Shape Bias and Robustness Evaluation via Cue Decomposition for Image Classification and Segmentation

• URL: http://arxiv.org/abs/2503.12453v1
• Date: Sun, 16 Mar 2025 11:17:03 GMT
• Title: Shape Bias and Robustness Evaluation via Cue Decomposition for Image Classification and Segmentation
• Authors: Edgar Heinert, Thomas Gottwald, Annika Mütze, Matthias Rottmann,
• Abstract summary: We study how deep neural networks perceive image content in terms of their biases towards different image cues, such as texture and shape.<n>We provide a new evaluation procedure consisting of 1) a cue-decomposition method that comprises two AI-free data pre-processing methods extracting shape and texture cues, respectively.<n>For application purposes we introduce a corresponding cue-decomposition robustness metric that allows for the estimation of the robustness of a DNN w.r.t. image corruptions.
• Score: 4.299840769087443
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Previous works studied how deep neural networks (DNNs) perceive image content in terms of their biases towards different image cues, such as texture and shape. Previous methods to measure shape and texture biases are typically style-transfer-based and limited to DNNs for image classification. In this work, we provide a new evaluation procedure consisting of 1) a cue-decomposition method that comprises two AI-free data pre-processing methods extracting shape and texture cues, respectively, and 2) a novel cue-decomposition shape bias evaluation metric that leverages the cue-decomposition data. For application purposes we introduce a corresponding cue-decomposition robustness metric that allows for the estimation of the robustness of a DNN w.r.t. image corruptions. In our numerical experiments, our findings for biases in image classification DNNs align with those of previous evaluation metrics. However, our cue-decomposition robustness metric shows superior results in terms of estimating the robustness of DNNs. Furthermore, our results for DNNs on the semantic segmentation datasets Cityscapes and ADE20k for the first time shed light into the biases of semantic segmentation DNNs.

Related papers

• Exploiting Diffusion Prior for Generalizable Dense Prediction [85.4563592053464]
  Recent advanced Text-to-Image (T2I) diffusion models are sometimes too imaginative for existing off-the-shelf dense predictors to estimate. We introduce DMP, a pipeline utilizing pre-trained T2I models as a prior for dense prediction tasks. Despite limited-domain training data, the approach yields faithful estimations for arbitrary images, surpassing existing state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2023-11-30T18:59:44Z)
• Information Bottleneck Analysis of Deep Neural Networks via Lossy Compression [37.69303106863453]
  The Information Bottleneck (IB) principle offers an information-theoretic framework for analyzing the training process of deep neural networks (DNNs) In this paper, we introduce a framework for conducting IB analysis of general NNs. We also perform IB analysis on a close-to-real-scale, which reveals new features of the MI dynamics.
  arXiv  Detail & Related papers  (2023-05-13T21:44:32Z)
• DeepDC: Deep Distance Correlation as a Perceptual Image Quality Evaluator [53.57431705309919]
  ImageNet pre-trained deep neural networks (DNNs) show notable transferability for building effective image quality assessment (IQA) models. We develop a novel full-reference IQA (FR-IQA) model based exclusively on pre-trained DNN features. We conduct comprehensive experiments to demonstrate the superiority of the proposed quality model on five standard IQA datasets.
  arXiv  Detail & Related papers  (2022-11-09T14:57:27Z)
• Decoupled Mixup for Generalized Visual Recognition [71.13734761715472]
  We propose a novel "Decoupled-Mixup" method to train CNN models for visual recognition. Our method decouples each image into discriminative and noise-prone regions, and then heterogeneously combines these regions to train CNN models. Experiment results show the high generalization performance of our method on testing data that are composed of unseen contexts.
  arXiv  Detail & Related papers  (2022-10-26T15:21:39Z)
• Invariant Content Synergistic Learning for Domain Generalization of Medical Image Segmentation [13.708239594165061]
  Deep convolution neural networks (DCNNs) often fail to maintain their robustness when confronting test data with the novel distribution. In this paper, we propose a method, named Invariant Content Synergistic Learning (ICSL), to improve the generalization ability of DCNNs.
  arXiv  Detail & Related papers  (2022-05-05T08:13:17Z)
• Action Recognition with Domain Invariant Features of Skeleton Image [25.519217340328442]
  We propose a novel CNN-based method with adversarial training for action recognition. We introduce a two-level domain adversarial learning to align the features of skeleton images from different view angles or subjects. It achieves competitive results compared with state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-11-19T08:05:54Z)
• Image Quality Assessment using Contrastive Learning [50.265638572116984]
  We train a deep Convolutional Neural Network (CNN) using a contrastive pairwise objective to solve the auxiliary problem. We show through extensive experiments that CONTRIQUE achieves competitive performance when compared to state-of-the-art NR image quality models. Our results suggest that powerful quality representations with perceptual relevance can be obtained without requiring large labeled subjective image quality datasets.
  arXiv  Detail & Related papers  (2021-10-25T21:01:00Z)
• Segmentation of Roads in Satellite Images using specially modified U-Net CNNs [0.0]
  The aim of this paper is to build an image classifier for satellite images of urban scenes that identifies the portions of the images in which a road is located. Unlike conventional computer vision algorithms, convolutional neural networks (CNNs) provide accurate and reliable results on this task.
  arXiv  Detail & Related papers  (2021-09-29T19:08:32Z)
• Consumer Image Quality Prediction using Recurrent Neural Networks for Spatial Pooling [13.750624267664156]
  We propose an image quality model that attempts to mimic the attention mechanism of human visual system (HVS) by using a recurrent neural network (RNN) The experimental study, conducted by using images with different resolutions from two recently published image quality datasets, indicates that the quality prediction accuracy of the proposed method is competitive against benchmark models representing the state-of-the-art, and the proposed method also performs consistently on different resolution versions of the same dataset.
  arXiv  Detail & Related papers  (2021-06-02T03:31:44Z)
• Image Restoration by Deep Projected GSURE [115.57142046076164]
  Ill-posed inverse problems appear in many image processing applications, such as deblurring and super-resolution. We propose a new image restoration framework that is based on minimizing a loss function that includes a "projected-version" of the Generalized SteinUnbiased Risk Estimator (GSURE) and parameterization of the latent image by a CNN.
  arXiv  Detail & Related papers  (2021-02-04T08:52:46Z)
• An Uncertainty-Driven GCN Refinement Strategy for Organ Segmentation [53.425900196763756]
  We propose a segmentation refinement method based on uncertainty analysis and graph convolutional networks. We employ the uncertainty levels of the convolutional network in a particular input volume to formulate a semi-supervised graph learning problem. We show that our method outperforms the state-of-the-art CRF refinement method by improving the dice score by 1% for the pancreas and 2% for spleen.
  arXiv  Detail & Related papers  (2020-12-06T18:55:07Z)
• Data Consistent CT Reconstruction from Insufficient Data with Learned Prior Images [70.13735569016752]
  We investigate the robustness of deep learning in CT image reconstruction by showing false negative and false positive lesion cases. We propose a data consistent reconstruction (DCR) method to improve their image quality, which combines the advantages of compressed sensing and deep learning. The efficacy of the proposed method is demonstrated in cone-beam CT with truncated data, limited-angle data and sparse-view data, respectively.
  arXiv  Detail & Related papers  (2020-05-20T13:30:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.