Hierarchical Contrastive Learning for Pattern-Generalizable Image Corruption Detection

• URL: http://arxiv.org/abs/2308.14061v1
• Date: Sun, 27 Aug 2023 10:03:48 GMT
• Title: Hierarchical Contrastive Learning for Pattern-Generalizable Image Corruption Detection
• Authors: Xin Feng, Yifeng Xu, Guangming Lu, Wenjie Pei
• Abstract summary: We develop a hierarchical contrastive learning framework to detect corrupted regions. A specialized hierarchical interaction mechanism is designed to facilitate the knowledge of contrastive learning in different scales. Our model has well generalization ability across different corruption patterns.
• Score: 40.04083743934034
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Effective image restoration with large-size corruptions, such as blind image inpainting, entails precise detection of corruption region masks which remains extremely challenging due to diverse shapes and patterns of corruptions. In this work, we present a novel method for automatic corruption detection, which allows for blind corruption restoration without known corruption masks. Specifically, we develop a hierarchical contrastive learning framework to detect corrupted regions by capturing the intrinsic semantic distinctions between corrupted and uncorrupted regions. In particular, our model detects the corrupted mask in a coarse-to-fine manner by first predicting a coarse mask by contrastive learning in low-resolution feature space and then refines the uncertain area of the mask by high-resolution contrastive learning. A specialized hierarchical interaction mechanism is designed to facilitate the knowledge propagation of contrastive learning in different scales, boosting the modeling performance substantially. The detected multi-scale corruption masks are then leveraged to guide the corruption restoration. Detecting corrupted regions by learning the contrastive distinctions rather than the semantic patterns of corruptions, our model has well generalization ability across different corruption patterns. Extensive experiments demonstrate following merits of our model: 1) the superior performance over other methods on both corruption detection and various image restoration tasks including blind inpainting and watermark removal, and 2) strong generalization across different corruption patterns such as graffiti, random noise or other image content. Codes and trained weights are available at https://github.com/xyfJASON/HCL .

Related papers

• Corruptions of Supervised Learning Problems: Typology and Mitigations [11.294508617469905]
  We develop a general theory of corruption from an information-theoretic perspective. We will focus here on changes in probability distributions. This work sheds light on complexities arising from joint and dependent corruptions on both labels and attributes.
  arXiv  Detail & Related papers  (2023-07-17T16:57:01Z)
• Frequency-Based Vulnerability Analysis of Deep Learning Models against Image Corruptions [48.34142457385199]
  We present MUFIA, an algorithm designed to identify the specific types of corruptions that can cause models to fail. We find that even state-of-the-art models trained to be robust against known common corruptions struggle against the low visibility-based corruptions crafted by MUFIA.
  arXiv  Detail & Related papers  (2023-06-12T15:19:13Z)
• Point-DAE: Denoising Autoencoders for Self-supervised Point Cloud Learning [54.51061298877896]
  We explore a more general denoising autoencoder for point cloud learning (Point-DAE) by investigating more types of corruptions beyond masking. Specifically, we degrade the point cloud with certain corruptions as input, and learn an encoder-decoder model to reconstruct the original point cloud from its corrupted version.
  arXiv  Detail & Related papers  (2022-11-13T08:02:03Z)
• Soft Diffusion: Score Matching for General Corruptions [84.26037497404195]
  We propose a new objective called Soft Score Matching that provably learns the score function for any linear corruption process. We show that our objective learns the gradient of the likelihood under suitable regularity conditions for the family of corruption processes. Our method achieves state-of-the-art FID score $1.85$ on CelebA-64, outperforming all previous linear diffusion models.
  arXiv  Detail & Related papers  (2022-09-12T17:45:03Z)
• Abutting Grating Illusion: Cognitive Challenge to Neural Network Models [4.031522806737616]
  We propose a novel corruption method based on the abutting grating illusion. The method destroys the gradient-defined boundaries and generates the perception of illusory contours using line gratings abutting each other. Various deep learning models are tested on the corruption, including models trained from scratch and 109 models pretrained with ImageNet or various data augmentation techniques.
  arXiv  Detail & Related papers  (2022-08-08T08:01:11Z)
• Self-Supervised Predictive Convolutional Attentive Block for Anomaly Detection [97.93062818228015]
  We propose to integrate the reconstruction-based functionality into a novel self-supervised predictive architectural building block. Our block is equipped with a loss that minimizes the reconstruction error with respect to the masked area in the receptive field. We demonstrate the generality of our block by integrating it into several state-of-the-art frameworks for anomaly detection on image and video.
  arXiv  Detail & Related papers  (2021-11-17T13:30:31Z)
• Improving robustness against common corruptions with frequency biased models [112.65717928060195]
  unseen image corruptions can cause a surprisingly large drop in performance. Image corruption types have different characteristics in the frequency spectrum and would benefit from a targeted type of data augmentation. We propose a new regularization scheme that minimizes the total variation (TV) of convolution feature-maps to increase high-frequency robustness.
  arXiv  Detail & Related papers  (2021-03-30T10:44:50Z)
• A simple way to make neural networks robust against diverse image corruptions [29.225922892332342]
  We show that a simple but properly tuned training with additive Gaussian and Speckle noise generalizes surprisingly well to unseen corruptions. An adversarial training of the recognition model against uncorrelated worst-case noise leads to an additional increase in performance.
  arXiv  Detail & Related papers  (2020-01-16T20:10:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.