Image Quality Assessment: Unifying Structure and Texture Similarity
- URL: http://arxiv.org/abs/2004.07728v3
- Date: Wed, 16 Dec 2020 12:56:44 GMT
- Title: Image Quality Assessment: Unifying Structure and Texture Similarity
- Authors: Keyan Ding, Kede Ma, Shiqi Wang, and Eero P. Simoncelli
- Abstract summary: We develop the first full-reference image quality model with explicit tolerance to texture resampling.
Using a convolutional neural network, we construct an injective and differentiable function that transforms images to overcomplete representations.
- Score: 38.05659069533254
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Objective measures of image quality generally operate by comparing pixels of
a "degraded" image to those of the original. Relative to human observers, these
measures are overly sensitive to resampling of texture regions (e.g., replacing
one patch of grass with another). Here, we develop the first full-reference
image quality model with explicit tolerance to texture resampling. Using a
convolutional neural network, we construct an injective and differentiable
function that transforms images to multi-scale overcomplete representations. We
demonstrate empirically that the spatial averages of the feature maps in this
representation capture texture appearance, in that they provide a set of
sufficient statistical constraints to synthesize a wide variety of texture
patterns. We then describe an image quality method that combines correlations
of these spatial averages ("texture similarity") with correlations of the
feature maps ("structure similarity"). The parameters of the proposed measure
are jointly optimized to match human ratings of image quality, while minimizing
the reported distances between subimages cropped from the same texture images.
Experiments show that the optimized method explains human perceptual scores,
both on conventional image quality databases, as well as on texture databases.
The measure also offers competitive performance on related tasks such as
texture classification and retrieval. Finally, we show that our method is
relatively insensitive to geometric transformations (e.g., translation and
dilation), without use of any specialized training or data augmentation. Code
is available at https://github.com/dingkeyan93/DISTS.
Related papers
- CSIM: A Copula-based similarity index sensitive to local changes for Image quality assessment [2.3874115898130865]
Image similarity metrics play an important role in computer vision applications, as they are used in image processing, computer vision and machine learning.
Existing metrics, such as PSNR, MSE, SSIM, ISSM and FSIM, often face limitations in terms of either speed, complexity or sensitivity to small changes in images.
A novel image similarity metric, namely CSIM, that combines real-time while being sensitive to subtle image variations is investigated in this paper.
arXiv Detail & Related papers (2024-10-02T10:46:05Z) - The applicability of transperceptual and deep learning approaches to the
study and mimicry of complex cartilaginous tissues [0.0]
Complex soft tissues, for example the knee meniscus, play a crucial role in mobility and joint health.
In order to design tissue substitutes, the internal architecture of the native tissue needs to be understood and replicated.
We explore a combined audio-visual approach - so called transperceptual - to generate artificial architectures mimicking the native ones.
arXiv Detail & Related papers (2022-11-21T08:51:52Z) - DeepDC: Deep Distance Correlation as a Perceptual Image Quality
Evaluator [53.57431705309919]
ImageNet pre-trained deep neural networks (DNNs) show notable transferability for building effective image quality assessment (IQA) models.
We develop a novel full-reference IQA (FR-IQA) model based exclusively on pre-trained DNN features.
We conduct comprehensive experiments to demonstrate the superiority of the proposed quality model on five standard IQA datasets.
arXiv Detail & Related papers (2022-11-09T14:57:27Z) - Multiscale Analysis for Improving Texture Classification [62.226224120400026]
This paper employs the Gaussian-Laplacian pyramid to treat different spatial frequency bands of a texture separately.
We aggregate features extracted from gray and color texture images using bio-inspired texture descriptors, information-theoretic measures, gray-level co-occurrence matrix features, and Haralick statistical features into a single feature vector.
arXiv Detail & Related papers (2022-04-21T01:32:22Z) - Locally Adaptive Structure and Texture Similarity for Image Quality
Assessment [33.58928017067797]
We describe a locally adaptive structure and texture similarity index for full-reference image quality assessment (IQA)
Specifically, we rely on a single statistical feature, namely the dispersion index, to localize texture regions at different scales.
The resulting A-DISTS is adapted to local image content, and is free of expensive human perceptual scores for supervised training.
arXiv Detail & Related papers (2021-10-16T09:19:56Z) - Controllable Person Image Synthesis with Spatially-Adaptive Warped
Normalization [72.65828901909708]
Controllable person image generation aims to produce realistic human images with desirable attributes.
We introduce a novel Spatially-Adaptive Warped Normalization (SAWN), which integrates a learned flow-field to warp modulation parameters.
We propose a novel self-training part replacement strategy to refine the pretrained model for the texture-transfer task.
arXiv Detail & Related papers (2021-05-31T07:07:44Z) - Ensembling with Deep Generative Views [72.70801582346344]
generative models can synthesize "views" of artificial images that mimic real-world variations, such as changes in color or pose.
Here, we investigate whether such views can be applied to real images to benefit downstream analysis tasks such as image classification.
We use StyleGAN2 as the source of generative augmentations and investigate this setup on classification tasks involving facial attributes, cat faces, and cars.
arXiv Detail & Related papers (2021-04-29T17:58:35Z) - Image Inpainting Guided by Coherence Priors of Semantics and Textures [62.92586889409379]
We introduce coherence priors between the semantics and textures which make it possible to concentrate on completing separate textures in a semantic-wise manner.
We also propose two coherence losses to constrain the consistency between the semantics and the inpainted image in terms of the overall structure and detailed textures.
arXiv Detail & Related papers (2020-12-15T02:59:37Z) - Texture image classification based on a pseudo-parabolic diffusion model [0.0]
The proposed approach is tested on the classification of well established benchmark texture databases and on a practical task of plant species recognition.
The good performance can be justified to a large extent by the ability of the pseudo-parabolic operator to smooth possibly noisy details inside homogeneous regions of the image.
arXiv Detail & Related papers (2020-11-14T00:04:07Z) - Learning Texture Invariant Representation for Domain Adaptation of
Semantic Segmentation [19.617821473205694]
It is challenging for a model trained with synthetic data to generalize to real data.
We diversity the texture of synthetic images using a style transfer algorithm.
We fine-tune the model with self-training to get direct supervision of the target texture.
arXiv Detail & Related papers (2020-03-02T13:11:54Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.