Shift-tolerant Perceptual Similarity Metric

• URL: http://arxiv.org/abs/2207.13686v1
• Date: Wed, 27 Jul 2022 17:55:04 GMT
• Title: Shift-tolerant Perceptual Similarity Metric
• Authors: Abhijay Ghildyal, Feng Liu
• Abstract summary: Existing perceptual similarity metrics assume an image and its reference are well aligned. This paper studies the effect of small misalignment, specifically a small shift between the input and reference image, on existing metrics. We develop a new deep neural network-based perceptual similarity metric.
• Score: 5.326626090397465
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing perceptual similarity metrics assume an image and its reference are well aligned. As a result, these metrics are often sensitive to a small alignment error that is imperceptible to the human eyes. This paper studies the effect of small misalignment, specifically a small shift between the input and reference image, on existing metrics, and accordingly develops a shift-tolerant similarity metric. This paper builds upon LPIPS, a widely used learned perceptual similarity metric, and explores architectural design considerations to make it robust against imperceptible misalignment. Specifically, we study a wide spectrum of neural network elements, such as anti-aliasing filtering, pooling, striding, padding, and skip connection, and discuss their roles in making a robust metric. Based on our studies, we develop a new deep neural network-based perceptual similarity metric. Our experiments show that our metric is tolerant to imperceptible shifts while being consistent with the human similarity judgment.

Related papers

• LipSim: A Provably Robust Perceptual Similarity Metric [56.03417732498859]
  We show the vulnerability of state-of-the-art perceptual similarity metrics based on an ensemble of ViT-based feature extractors to adversarial attacks. We then propose a framework to train a robust perceptual similarity metric called LipSim with provable guarantees. LipSim provides guarded areas around each data point and certificates for all perturbations within an $ell$ ball.
  arXiv  Detail & Related papers  (2023-10-27T16:59:51Z)
• A Geometrical Approach to Evaluate the Adversarial Robustness of Deep Neural Networks [52.09243852066406]
  Adversarial Converging Time Score (ACTS) measures the converging time as an adversarial robustness metric. We validate the effectiveness and generalization of the proposed ACTS metric against different adversarial attacks on the large-scale ImageNet dataset.
  arXiv  Detail & Related papers  (2023-10-10T09:39:38Z)
• R-LPIPS: An Adversarially Robust Perceptual Similarity Metric [71.33812578529006]
  We propose the Robust Learned Perceptual Image Patch Similarity (R-LPIPS) metric. R-LPIPS is a new metric that leverages adversarially trained deep features. We demonstrate the superiority of R-LPIPS compared to the classical LPIPS metric.
  arXiv  Detail & Related papers  (2023-07-27T19:11:31Z)
• DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Data [43.247597420676044]
  Current perceptual similarity metrics operate at the level of pixels and patches. These metrics compare images in terms of their low-level colors and textures, but fail to capture mid-level similarities and differences in image layout, object pose, and semantic content. We develop a perceptual metric that assesses images holistically.
  arXiv  Detail & Related papers  (2023-06-15T17:59:50Z)
• Attacking Perceptual Similarity Metrics [5.326626090397465]
  We systematically examine the robustness of similarity metrics to imperceptible adversarial perturbations. We first show that all metrics in our study are susceptible to perturbations generated via common adversarial attacks. Next, we attack the widely adopted LPIPS metric using spatial-transformation-based adversarial perturbations.
  arXiv  Detail & Related papers  (2023-05-15T17:55:04Z)
• Self-similarity Driven Scale-invariant Learning for Weakly Supervised Person Search [66.95134080902717]
  We propose a novel one-step framework, named Self-similarity driven Scale-invariant Learning (SSL) We introduce a Multi-scale Exemplar Branch to guide the network in concentrating on the foreground and learning scale-invariant features. Experiments on PRW and CUHK-SYSU databases demonstrate the effectiveness of our method.
  arXiv  Detail & Related papers  (2023-02-25T04:48:11Z)
• Identifying and Mitigating Flaws of Deep Perceptual Similarity Metrics [1.484528358552186]
  This work investigates the benefits and flaws of the Deep Perceptual Similarity (DPS) metric. The metrics are analyzed in-depth to understand the strengths and weaknesses of the metrics. This work contributes with new insights into the flaws of DPS, and further suggests improvements to the metrics.
  arXiv  Detail & Related papers  (2022-07-06T08:28:39Z)
• Modulated Contrast for Versatile Image Synthesis [60.304183493234376]
  MoNCE is a versatile metric that introduces image contrast to learn a calibrated metric for the perception of multifaceted inter-image distances. We introduce optimal transport in MoNCE to modulate the pushing force of negative samples collaboratively across multiple contrastive objectives.
  arXiv  Detail & Related papers  (2022-03-17T14:03:46Z)
• Feature Space Targeted Attacks by Statistic Alignment [74.40447383387574]
  Feature space targeted attacks perturb images by modulating their intermediate feature maps. The current choice of pixel-wise Euclidean Distance to measure the discrepancy is questionable because it unreasonably imposes a spatial-consistency constraint on the source and target features. We propose two novel approaches called Pair-wise Alignment Attack and Global-wise Alignment Attack, which attempt to measure similarities between feature maps by high-order statistics.
  arXiv  Detail & Related papers  (2021-05-25T03:46:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.