Image Quality Assessment: Investigating Causal Perceptual Effects with Abductive Counterfactual Inference

• URL: http://arxiv.org/abs/2412.16939v1
• Date: Sun, 22 Dec 2024 09:17:57 GMT
• Title: Image Quality Assessment: Investigating Causal Perceptual Effects with Abductive Counterfactual Inference
• Authors: Wenhao Shen, Mingliang Zhou, Yu Chen, Xuekai Wei, Jun Luo, Huayan Pu, Weijia Jia,
• Abstract summary: Existing full-reference image quality assessment (FR-IQA) methods often fail to capture the complex causal mechanisms that underlie human perceptual responses to image distortions. We propose an FR-IQA method based on abductive counterfactual inference to investigate the causal relationships between deep network features and perceptual distortions.
• Score: 22.65765161695905
• License:
• Abstract: Existing full-reference image quality assessment (FR-IQA) methods often fail to capture the complex causal mechanisms that underlie human perceptual responses to image distortions, limiting their ability to generalize across diverse scenarios. In this paper, we propose an FR-IQA method based on abductive counterfactual inference to investigate the causal relationships between deep network features and perceptual distortions. First, we explore the causal effects of deep features on perception and integrate causal reasoning with feature comparison, constructing a model that effectively handles complex distortion types across different IQA scenarios. Second, the analysis of the perceptual causal correlations of our proposed method is independent of the backbone architecture and thus can be applied to a variety of deep networks. Through abductive counterfactual experiments, we validate the proposed causal relationships, confirming the model's superior perceptual relevance and interpretability of quality scores. The experimental results demonstrate the robustness and effectiveness of the method, providing competitive quality predictions across multiple benchmarks. The source code is available at https://anonymous.4open.science/r/DeepCausalQuality-25BC.

Related papers

• Causality can systematically address the monsters under the bench(marks) [64.36592889550431]
  Benchmarks are plagued by various biases, artifacts, or leakage. Models may behave unreliably due to poorly explored failure modes. causality offers an ideal framework to systematically address these challenges.
  arXiv  Detail & Related papers  (2025-02-07T17:01:37Z)
• Causal Perception Inspired Representation Learning for Trustworthy Image Quality Assessment [2.290956583394892]
  We propose to build a trustworthy IQA model via Causal Perception inspired Representation Learning (CPRL) CPRL serves as the causation of the subjective quality label, which is invariant to the imperceptible adversarial perturbations. Experiments on four benchmark databases show that the proposed CPRL method outperforms many state-of-the-art adversarial defense methods.
  arXiv  Detail & Related papers  (2024-04-30T13:55:30Z)
• When No-Reference Image Quality Models Meet MAP Estimation in Diffusion Latents [92.45867913876691]
  No-reference image quality assessment (NR-IQA) models can effectively quantify perceived image quality. We show that NR-IQA models can be plugged into the maximum a posteriori (MAP) estimation framework for image enhancement.
  arXiv  Detail & Related papers  (2024-03-11T03:35:41Z)
• Advancing Counterfactual Inference through Nonlinear Quantile Regression [77.28323341329461]
  We propose a framework for efficient and effective counterfactual inference implemented with neural networks. The proposed approach enhances the capacity to generalize estimated counterfactual outcomes to unseen data. Empirical results conducted on multiple datasets offer compelling support for our theoretical assertions.
  arXiv  Detail & Related papers  (2023-06-09T08:30:51Z)
• Fairness Increases Adversarial Vulnerability [50.90773979394264]
  This paper shows the existence of a dichotomy between fairness and robustness, and analyzes when achieving fairness decreases the model robustness to adversarial samples. Experiments on non-linear models and different architectures validate the theoretical findings in multiple vision domains. The paper proposes a simple, yet effective, solution to construct models achieving good tradeoffs between fairness and robustness.
  arXiv  Detail & Related papers  (2022-11-21T19:55:35Z)
• How Tempering Fixes Data Augmentation in Bayesian Neural Networks [22.188535244056016]
  We show that tempering implicitly reduces the misspecification arising from modeling augmentations as i.i.d. data. The temperature mimics the role of the effective sample size, reflecting the gain in information provided by the augmentations.
  arXiv  Detail & Related papers  (2022-05-27T11:06:56Z)
• Residual Error: a New Performance Measure for Adversarial Robustness [85.0371352689919]
  A major challenge that limits the wide-spread adoption of deep learning has been their fragility to adversarial attacks. This study presents the concept of residual error, a new performance measure for assessing the adversarial robustness of a deep neural network. Experimental results using the case of image classification demonstrate the effectiveness and efficacy of the proposed residual error metric.
  arXiv  Detail & Related papers  (2021-06-18T16:34:23Z)
• Contrastive Reasoning in Neural Networks [26.65337569468343]
  Inference built on features that identify causal class dependencies is termed as feed-forward inference. In this paper, we formalize the structure of contrastive reasoning and propose a methodology to extract a neural network's notion of contrast. We demonstrate the value of contrastively recognizing images under distortions by reporting an improvement of 3.47%, 2.56%, and 5.48% in average accuracy.
  arXiv  Detail & Related papers  (2021-03-23T05:54:36Z)
• Verifying the Causes of Adversarial Examples [5.381050729919025]
  The robustness of neural networks is challenged by adversarial examples that contain almost imperceptible perturbations to inputs. We present a collection of potential causes of adversarial examples and verify (or partially verify) them through carefully-designed controlled experiments. Our experiment results show that geometric factors tend to be more direct causes and statistical factors magnify the phenomenon.
  arXiv  Detail & Related papers  (2020-10-19T16:17:20Z)
• Ramifications of Approximate Posterior Inference for Bayesian Deep Learning in Adversarial and Out-of-Distribution Settings [7.476901945542385]
  We show that Bayesian deep learning models on certain occasions marginally outperform conventional neural networks. Preliminary investigations indicate the potential inherent role of bias due to choices of initialisation, architecture or activation functions.
  arXiv  Detail & Related papers  (2020-09-03T16:58:15Z)
• A Critical View of the Structural Causal Model [89.43277111586258]
  We show that one can identify the cause and the effect without considering their interaction at all. We propose a new adversarial training method that mimics the disentangled structure of the causal model. Our multidimensional method outperforms the literature methods on both synthetic and real world datasets.
  arXiv  Detail & Related papers  (2020-02-23T22:52:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.