Sensitivity-Informed Augmentation for Robust Segmentation

• URL: http://arxiv.org/abs/2406.01425v4
• Date: Sun, 16 Jun 2024 11:59:46 GMT
• Title: Sensitivity-Informed Augmentation for Robust Segmentation
• Authors: Laura Zheng, Wenjie Wei, Tony Wu, Jacob Clements, Shreelekha Revankar, Andre Harrison, Yu Shen, Ming C. Lin,
• Abstract summary: Internal noises such as variations in camera quality or lens distortion can affect the performance of segmentation models. We present an efficient, adaptable, and gradient-free method to enhance the robustness of learning-based segmentation models across training.
• Score: 21.609070498399863
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Segmentation is an integral module in many visual computing applications such as virtual try-on, medical imaging, autonomous driving, and agricultural automation. These applications often involve either widespread consumer use or highly variable environments, both of which can degrade the quality of visual sensor data, whether from a common mobile phone or an expensive satellite imaging camera. In addition to external noises like user difference or weather conditions, internal noises such as variations in camera quality or lens distortion can affect the performance of segmentation models during both development and deployment. In this work, we present an efficient, adaptable, and gradient-free method to enhance the robustness of learning-based segmentation models across training. First, we introduce a novel adaptive sensitivity analysis (ASA) using Kernel Inception Distance (KID) on basis perturbations to benchmark perturbation sensitivity of pre-trained segmentation models. Then, we model the sensitivity curve using the adaptive SA and sample perturbation hyperparameter values accordingly. Finally, we conduct adversarial training with the selected perturbation values and dynamically re-evaluate robustness during online training. Our method, implemented end-to-end with minimal fine-tuning required, consistently outperforms state-of-the-art data augmentation techniques for segmentation. It shows significant improvement in both clean data evaluation and real-world adverse scenario evaluation across various segmentation datasets used in visual computing and computer graphics applications.

Related papers

• Adaptive Domain Learning for Cross-domain Image Denoising [57.4030317607274]
  We present a novel adaptive domain learning scheme for cross-domain image denoising. We use existing data from different sensors (source domain) plus a small amount of data from the new sensor (target domain) The ADL training scheme automatically removes the data in the source domain that are harmful to fine-tuning a model for the target domain. Also, we introduce a modulation module to adopt sensor-specific information (sensor type and ISO) to understand input data for image denoising.
  arXiv  Detail & Related papers  (2024-11-03T08:08:26Z)
• BEVal: A Cross-dataset Evaluation Study of BEV Segmentation Models for Autonomous Driving [3.4113606473878386]
  We conduct a comprehensive cross-dataset evaluation of state-of-the-art BEV segmentation models. We investigate the influence of different sensors, such as cameras and LiDAR, on the models' ability to generalize.
  arXiv  Detail & Related papers  (2024-08-29T07:49:31Z)
• DeltaNN: Assessing the Impact of Computational Environment Parameters on the Performance of Image Recognition Models [2.379078565066793]
  Failure in real-time image recognition tasks can occur due to sub-optimal mapping on hardware accelerators. We present a differential testing framework, DeltaNN, that allows us to assess the impact of different computational environment parameters on the performance of image recognition models.
  arXiv  Detail & Related papers  (2023-06-05T23:07:01Z)
• A quality assurance framework for real-time monitoring of deep learning segmentation models in radiotherapy [3.5752677591512487]
  This work uses cardiac substructure segmentation as an example task to establish a quality assurance framework. A benchmark dataset consisting of Computed Tomography (CT) images along with manual cardiac delineations of 241 patients was collected. An image domain shift detector was developed by utilizing a trained Denoising autoencoder (DAE) and two hand-engineered features. A regression model was trained to predict the per-patient segmentation accuracy, measured by Dice similarity coefficient (DSC)
  arXiv  Detail & Related papers  (2023-05-19T14:51:05Z)
• A Comprehensive Study of Image Classification Model Sensitivity to Foregrounds, Backgrounds, and Visual Attributes [58.633364000258645]
  We call this dataset RIVAL10 consisting of roughly $26k$ instances over $10$ classes. We evaluate the sensitivity of a broad set of models to noise corruptions in foregrounds, backgrounds and attributes. In our analysis, we consider diverse state-of-the-art architectures (ResNets, Transformers) and training procedures (CLIP, SimCLR, DeiT, Adversarial Training)
  arXiv  Detail & Related papers  (2022-01-26T06:31:28Z)
• DAPPER: Label-Free Performance Estimation after Personalization for Heterogeneous Mobile Sensing [95.18236298557721]
  We present DAPPER (Domain AdaPtation Performance EstimatoR) that estimates the adaptation performance in a target domain with unlabeled target data. Our evaluation with four real-world sensing datasets compared against six baselines shows that DAPPER outperforms the state-of-the-art baseline by 39.8% in estimation accuracy.
  arXiv  Detail & Related papers  (2021-11-22T08:49:33Z)
• An automatic differentiation system for the age of differential privacy [65.35244647521989]
  Tritium is an automatic differentiation-based sensitivity analysis framework for differentially private (DP) machine learning (ML) We introduce Tritium, an automatic differentiation-based sensitivity analysis framework for differentially private (DP) machine learning (ML)
  arXiv  Detail & Related papers  (2021-09-22T08:07:42Z)
• Comparison of end-to-end neural network architectures and data augmentation methods for automatic infant motility assessment using wearable sensors [7.003610369186623]
  This study investigates the use of different end-to-end neural network architectures for processing infant motility data from wearable sensors. The experiments are conducted using a data-set of multi-sensor movement recordings from 7-month-old infants.
  arXiv  Detail & Related papers  (2021-07-02T14:02:05Z)
• Cycle and Semantic Consistent Adversarial Domain Adaptation for Reducing Simulation-to-Real Domain Shift in LiDAR Bird's Eye View [110.83289076967895]
  We present a BEV domain adaptation method based on CycleGAN that uses prior semantic classification in order to preserve the information of small objects of interest during the domain adaptation process. The quality of the generated BEVs has been evaluated using a state-of-the-art 3D object detection framework at KITTI 3D Object Detection Benchmark.
  arXiv  Detail & Related papers  (2021-04-22T12:47:37Z)
• Firearm Detection via Convolutional Neural Networks: Comparing a Semantic Segmentation Model Against End-to-End Solutions [68.8204255655161]
  Threat detection of weapons and aggressive behavior from live video can be used for rapid detection and prevention of potentially deadly incidents. One way for achieving this is through the use of artificial intelligence and, in particular, machine learning for image analysis. We compare a traditional monolithic end-to-end deep learning model and a previously proposed model based on an ensemble of simpler neural networks detecting fire-weapons via semantic segmentation.
  arXiv  Detail & Related papers  (2020-12-17T15:19:29Z)
• Domain-invariant Similarity Activation Map Contrastive Learning for Retrieval-based Long-term Visual Localization [30.203072945001136]
  In this work, a general architecture is first formulated probabilistically to extract domain invariant feature through multi-domain image translation. And then a novel gradient-weighted similarity activation mapping loss (Grad-SAM) is incorporated for finer localization with high accuracy. Extensive experiments have been conducted to validate the effectiveness of the proposed approach on the CMUSeasons dataset. Our performance is on par with or even outperforms the state-of-the-art image-based localization baselines in medium or high precision.
  arXiv  Detail & Related papers  (2020-09-16T14:43:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.