Recognizing Instagram Filtered Images with Feature De-stylization

• URL: http://arxiv.org/abs/1912.13000v1
• Date: Mon, 30 Dec 2019 16:48:16 GMT
• Title: Recognizing Instagram Filtered Images with Feature De-stylization
• Authors: Zhe Wu, Zuxuan Wu, Bharat Singh, Larry S. Davis
• Abstract summary: This paper presents a study on how popular pretrained models are affected by commonly used Instagram filters. Our analysis suggests that simple structure preserving filters which only alter the global appearance of an image can lead to large differences in the convolutional feature space. We introduce a lightweight de-stylization module that predicts parameters used for scaling and shifting feature maps to "undo" the changes incurred by filters.
• Score: 81.38905784617089
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep neural networks have been shown to suffer from poor generalization when small perturbations are added (like Gaussian noise), yet little work has been done to evaluate their robustness to more natural image transformations like photo filters. This paper presents a study on how popular pretrained models are affected by commonly used Instagram filters. To this end, we introduce ImageNet-Instagram, a filtered version of ImageNet, where 20 popular Instagram filters are applied to each image in ImageNet. Our analysis suggests that simple structure preserving filters which only alter the global appearance of an image can lead to large differences in the convolutional feature space. To improve generalization, we introduce a lightweight de-stylization module that predicts parameters used for scaling and shifting feature maps to "undo" the changes incurred by filters, inverting the process of style transfer tasks. We further demonstrate the module can be readily plugged into modern CNN architectures together with skip connections. We conduct extensive studies on ImageNet-Instagram, and show quantitatively and qualitatively, that the proposed module, among other things, can effectively improve generalization by simply learning normalization parameters without retraining the entire network, thus recovering the alterations in the feature space caused by the filters.

Related papers

• Catch-Up Mix: Catch-Up Class for Struggling Filters in CNN [15.3232203753165]
  Deep learning models often face challenges related to complexity and overfitting. One notable concern is that the model often relies heavily on a limited subset of filters for making predictions. We present a novel method called Catch-up Mix, which provides learning opportunities to a wide range of filters during training.
  arXiv  Detail & Related papers  (2024-01-24T02:42:50Z)
• Uncovering the Disentanglement Capability in Text-to-Image Diffusion Models [60.63556257324894]
  A key desired property of image generative models is the ability to disentangle different attributes. We propose a simple, light-weight image editing algorithm where the mixing weights of the two text embeddings are optimized for style matching and content preservation. Experiments show that the proposed method can modify a wide range of attributes, with the performance outperforming diffusion-model-based image-editing algorithms.
  arXiv  Detail & Related papers  (2022-12-16T19:58:52Z)
• Gabor is Enough: Interpretable Deep Denoising with a Gabor Synthesis Dictionary Prior [6.297103076360578]
  Gabor-like filters have been observed in the early layers of CNN classifiers and throughout low-level image processing networks. In this work, we take this observation to the extreme and explicitly constrain the filters of a natural-image denoising CNN to be learned 2D real Gabor filters. We find that the proposed network (GDLNet) can achieve near state-of-the-art denoising performance amongst popular fully convolutional neural networks.
  arXiv  Detail & Related papers  (2022-04-23T22:21:54Z)
• Patch-wise Contrastive Style Learning for Instagram Filter Removal [3.867363075280544]
  Social media filters are one of the most common resources of various corruptions and perturbations for real-world visual analysis applications. We introduce Contrastive Instagram Filter Removal Network (CIFR), which enhances this idea for Instagram filter removal by employing a novel multi-layer patch-wise contrastive style learning mechanism.
  arXiv  Detail & Related papers  (2022-04-15T14:38:28Z)
• Unsharp Mask Guided Filtering [53.14430987860308]
  The goal of this paper is guided image filtering, which emphasizes the importance of structure transfer during filtering. We propose a new and simplified formulation of the guided filter inspired by unsharp masking. Our formulation enjoys a filtering prior to a low-pass filter and enables explicit structure transfer by estimating a single coefficient.
  arXiv  Detail & Related papers  (2021-06-02T19:15:34Z)
• Graph Neural Networks with Adaptive Frequency Response Filter [55.626174910206046]
  We develop a graph neural network framework AdaGNN with a well-smooth adaptive frequency response filter. We empirically validate the effectiveness of the proposed framework on various benchmark datasets.
  arXiv  Detail & Related papers  (2021-04-26T19:31:21Z)
• Instagram Filter Removal on Fashionable Images [2.1485350418225244]
  We introduce Instagram Filter Removal Network (IFRNet) to mitigate the effects of image filters for social media analysis applications. Experiments demonstrate IFRNet outperforms all compared methods in quantitative and qualitative comparisons. We present the filter classification performance of our proposed model, and analyze the dominant color estimation on the images unfiltered by all compared methods.
  arXiv  Detail & Related papers  (2021-04-11T18:44:43Z)
• Look here! A parametric learning based approach to redirect visual attention [49.609412873346386]
  We introduce an automatic method to make an image region more attention-capturing via subtle image edits. Our model predicts a distinct set of global parametric transformations to be applied to the foreground and background image regions. Our edits enable inference at interactive rates on any image size, and easily generalize to videos.
  arXiv  Detail & Related papers  (2020-08-12T16:08:36Z)
• Filter Grafting for Deep Neural Networks: Reason, Method, and Cultivation [86.91324735966766]
  Filter is the key component in modern convolutional neural networks (CNNs) In this paper, we introduce filter grafting (textbfMethod) to achieve this goal. We develop a novel criterion to measure the information of filters and an adaptive weighting strategy to balance the grafted information among networks.
  arXiv  Detail & Related papers  (2020-04-26T08:36:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.