Disentangling Image Distortions in Deep Feature Space
- URL: http://arxiv.org/abs/2002.11409v2
- Date: Tue, 16 Jun 2020 13:04:12 GMT
- Title: Disentangling Image Distortions in Deep Feature Space
- Authors: Simone Bianco, Luigi Celona, Paolo Napoletano
- Abstract summary: We take a step in the direction of a broader understanding of perceptual similarity by analyzing the capability of deep visual representations to intrinsically characterize different types of image distortions.
A dimension-reduced representation of the features extracted from a given layer permits to efficiently separate types of distortions in the feature space.
Each network layer exhibits a different ability to separate between different types of distortions, and this ability varies according to the network architecture.
- Score: 20.220653544354285
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Previous literature suggests that perceptual similarity is an emergent
property shared across deep visual representations. Experiments conducted on a
dataset of human-judged image distortions have proven that deep features
outperform classic perceptual metrics. In this work we take a further step in
the direction of a broader understanding of such property by analyzing the
capability of deep visual representations to intrinsically characterize
different types of image distortions. To this end, we firstly generate a number
of synthetically distorted images and then we analyze the features extracted by
different layers of different Deep Neural Networks. We observe that a
dimension-reduced representation of the features extracted from a given layer
permits to efficiently separate types of distortions in the feature space.
Moreover, each network layer exhibits a different ability to separate between
different types of distortions, and this ability varies according to the
network architecture. Finally, we evaluate the exploitation of features taken
from the layer that better separates image distortions for: i)
reduced-reference image quality assessment, and ii) distortion types and
severity levels characterization on both single and multiple distortion
databases. Results achieved on both tasks suggest that deep visual
representations can be unsupervisedly employed to efficiently characterize
various image distortions.
Related papers
- Super Consistency of Neural Network Landscapes and Learning Rate Transfer [72.54450821671624]
We study the landscape through the lens of the loss Hessian.
We find that certain spectral properties under $mu$P are largely independent of the size of the network.
We show that in the Neural Tangent Kernel (NTK) and other scaling regimes, the sharpness exhibits very different dynamics at different scales.
arXiv Detail & Related papers (2024-02-27T12:28:01Z) - Image Deblurring by Exploring In-depth Properties of Transformer [86.7039249037193]
We leverage deep features extracted from a pretrained vision transformer (ViT) to encourage recovered images to be sharp without sacrificing the performance measured by the quantitative metrics.
By comparing the transformer features between recovered image and target one, the pretrained transformer provides high-resolution blur-sensitive semantic information.
One regards the features as vectors and computes the discrepancy between representations extracted from recovered image and target one in Euclidean space.
arXiv Detail & Related papers (2023-03-24T14:14:25Z) - Do Different Deep Metric Learning Losses Lead to Similar Learned
Features? [4.043200001974071]
We compare 14 pretrained models from a recent study and find that, even though all models perform similarly, different loss functions can guide the model to learn different features.
Our analysis also shows that some seemingly irrelevant properties can have significant influence on the resulting embedding.
arXiv Detail & Related papers (2022-05-05T15:07:19Z) - Enhancing Photorealism Enhancement [83.88433283714461]
We present an approach to enhancing the realism of synthetic images using a convolutional network.
We analyze scene layout distributions in commonly used datasets and find that they differ in important ways.
We report substantial gains in stability and realism in comparison to recent image-to-image translation methods.
arXiv Detail & Related papers (2021-05-10T19:00:49Z) - Projected Distribution Loss for Image Enhancement [15.297569497776374]
We show that aggregating 1D-Wasserstein distances between CNN activations is more reliable than the existing approaches.
In imaging applications such as denoising, super-resolution, demosaicing, deblurring and JPEG artifact removal, the proposed learning loss outperforms the current state-of-the-art on reference-based perceptual losses.
arXiv Detail & Related papers (2020-12-16T22:13:03Z) - SIR: Self-supervised Image Rectification via Seeing the Same Scene from
Multiple Different Lenses [82.56853587380168]
We propose a novel self-supervised image rectification (SIR) method based on an important insight that the rectified results of distorted images of the same scene from different lens should be the same.
We leverage a differentiable warping module to generate the rectified images and re-distorted images from the distortion parameters.
Our method achieves comparable or even better performance than the supervised baseline method and representative state-of-the-art methods.
arXiv Detail & Related papers (2020-11-30T08:23:25Z) - Restoring Spatially-Heterogeneous Distortions using Mixture of Experts
Network [11.048041466120589]
We introduce a spatially-heterogeneous distortion dataset in which multiple corruptions are applied to the different locations of each image.
Motivated by the multi-task learning, we design our network to have multiple paths that learn both common and distortion-specific representations.
Our model is effective for restoring real-world distortions and we experimentally verify that our method outperforms other models designed to manage both single distortion and multiple distortions.
arXiv Detail & Related papers (2020-09-30T11:06:38Z) - Exploring Intensity Invariance in Deep Neural Networks for Brain Image
Registration [0.0]
We investigate the effect of intensity distribution among input image pairs for deep learning-based image registration methods.
Deep learning models trained with structure similarity-based loss seems to perform better for both datasets.
arXiv Detail & Related papers (2020-09-21T17:49:03Z) - Generative Hierarchical Features from Synthesizing Images [65.66756821069124]
We show that learning to synthesize images can bring remarkable hierarchical visual features that are generalizable across a wide range of applications.
The visual feature produced by our encoder, termed as Generative Hierarchical Feature (GH-Feat), has strong transferability to both generative and discriminative tasks.
arXiv Detail & Related papers (2020-07-20T18:04:14Z) - Disentangle Perceptual Learning through Online Contrastive Learning [16.534353501066203]
Pursuing realistic results according to human visual perception is the central concern in the image transformation tasks.
In this paper, we argue that, among the features representation from the pre-trained classification network, only limited dimensions are related to human visual perception.
Under such an assumption, we try to disentangle the perception-relevant dimensions from the representation through our proposed online contrastive learning.
arXiv Detail & Related papers (2020-06-24T06:48:38Z) - Shallow Feature Based Dense Attention Network for Crowd Counting [103.67446852449551]
We propose a Shallow feature based Dense Attention Network (SDANet) for crowd counting from still images.
Our method outperforms other existing methods by a large margin, as is evident from a remarkable 11.9% Mean Absolute Error (MAE) drop of our SDANet.
arXiv Detail & Related papers (2020-06-17T13:34:42Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.