Looks Too Good To Be True: An Information-Theoretic Analysis of Hallucinations in Generative Restoration Models

• URL: http://arxiv.org/abs/2405.16475v3
• Date: Fri, 25 Oct 2024 19:40:30 GMT
• Title: Looks Too Good To Be True: An Information-Theoretic Analysis of Hallucinations in Generative Restoration Models
• Authors: Regev Cohen, Idan Kligvasser, Ehud Rivlin, Daniel Freedman,
• Abstract summary: generative models are capable of producing results often visually indistinguishable from real data. They also exhibit a growing tendency to generate hallucinations - realistic-looking details that do not exist in the ground truth images. This paper investigates this phenomenon through the lens of information theory, revealing a fundamental tradeoff between uncertainty and perception.
• Score: 13.605340325383452
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The pursuit of high perceptual quality in image restoration has driven the development of revolutionary generative models, capable of producing results often visually indistinguishable from real data. However, as their perceptual quality continues to improve, these models also exhibit a growing tendency to generate hallucinations - realistic-looking details that do not exist in the ground truth images. Hallucinations in these models create uncertainty about their reliability, raising major concerns about their practical application. This paper investigates this phenomenon through the lens of information theory, revealing a fundamental tradeoff between uncertainty and perception. We rigorously analyze the relationship between these two factors, proving that the global minimal uncertainty in generative models grows in tandem with perception. In particular, we define the inherent uncertainty of the restoration problem and show that attaining perfect perceptual quality entails at least twice this uncertainty. Additionally, we establish a relation between distortion, uncertainty and perception, through which we prove the aforementioned uncertainly-perception tradeoff induces the well-known perception-distortion tradeoff. We demonstrate our theoretical findings through experiments with super-resolution and inpainting algorithms. This work uncovers fundamental limitations of generative models in achieving both high perceptual quality and reliable predictions for image restoration. Thus, we aim to raise awareness among practitioners about this inherent tradeoff, empowering them to make informed decisions and potentially prioritize safety over perceptual performance.

Related papers

• DR.Experts: Differential Refinement of Distortion-Aware Experts for Blind Image Quality Assessment [9.682871853344226]
  We introduce DR.Experts, a novel prior-driven BIQA framework designed to explicitly incorporate distortion priors.<n> DR.Experts begins by leveraging a degradation-aware vision-language model to obtain distortion-specific priors.<n>The refined priors, along with semantics and bridging representation, are then fused by a proposed mixture-of-experts style module.<n>This mechanism weights each distortion-specific feature as per its perceptual impact, ensuring that the final quality prediction aligns with human perception.
  arXiv  Detail & Related papers  (2026-02-10T08:41:40Z)
• Counterfactual Visual Explanation via Causally-Guided Adversarial Steering [24.74447218872395]
  Recent work on counterfactual visual explanations has contributed to making artificial intelligence models more explainable.<n>These approaches neglect the causal relationships and the spurious correlations behind the image generation process.<n>We introduce a novel framework CECAS, which first leverages a causally-guided adversarial method to generate counterfactual explanations.
  arXiv  Detail & Related papers  (2025-07-14T03:36:36Z)
• On the Effect of Uncertainty on Layer-wise Inference Dynamics [21.022511880831605]
  We show that uncertainty does not seem to affect inference dynamics.<n>We balance this finding by showing evidence that more competent models may learn to process uncertainty differently.
  arXiv  Detail & Related papers  (2025-07-09T10:30:09Z)
• Seeing is Believing? Mitigating OCR Hallucinations in Multimodal Large Language Models [22.43132625619281]
  We propose KIE-HVQA, the first benchmark dedicated to evaluating OCR hallucination in degraded document understanding.<n>This dataset includes test samples spanning identity cards and invoices, with simulated real-world degradations for OCR reliability.<n>Experiments on Qwen2.5-VL demonstrate that our 7B- parameter model achieves a 22% absolute improvement in hallucination-free accuracy over GPT-4o.
  arXiv  Detail & Related papers  (2025-06-25T06:44:07Z)
• Delusions of Large Language Models [62.43923767408462]
  Large Language Models often generate factually incorrect but plausible outputs, known as hallucinations. We identify a more insidious phenomenon, LLM delusion, defined as high belief hallucinations, incorrect outputs with abnormally high confidence, making them harder to detect and mitigate.
  arXiv  Detail & Related papers  (2025-03-09T17:59:16Z)
• Epistemic Uncertainty for Generated Image Detection [107.62647907393377]
  We introduce a novel framework for AI-generated image detection through epistemic uncertainty, aiming to address critical security concerns in the era of generative models.<n>Our key insight stems from the observation that distributional discrepancies between training and testing data manifest distinctively in the epistemic uncertainty space of machine learning models.
  arXiv  Detail & Related papers  (2024-12-08T11:32:25Z)
• Uncertainty-Informed Volume Visualization using Implicit Neural Representation [6.909370175721755]
  We propose uncertainty-aware implicit neural representations to model scalar field data sets. We evaluate the effectiveness of two principled deep uncertainty estimation techniques. Our work makes it suitable for robustly analyzing and visualizing real-world scientific volumetric data sets.
  arXiv  Detail & Related papers  (2024-08-12T09:14:23Z)
• Visual Analysis of Prediction Uncertainty in Neural Networks for Deep Image Synthesis [3.09988520562118]
  It is imperative to comprehend the quality, confidence, robustness, and uncertainty associated with their prediction. A thorough understanding of these quantities produces actionable insights that help application scientists make informed decisions. This contribution demonstrates how the prediction uncertainty and sensitivity of DNNs can be estimated efficiently using various methods.
  arXiv  Detail & Related papers  (2024-05-22T20:01:31Z)
• Uncertainty in latent representations of variational autoencoders optimized for visual tasks [4.919240908498475]
  We study uncertainty representations in latent representations of variational auto-encoders (VAEs) We show how a novel approach which we call explaining-away variational auto-encoders (EA-VAEs) fixes these issues. EA-VAEs may prove useful both as models of perception in computational neuroscience and as inference tools in computer vision.
  arXiv  Detail & Related papers  (2024-04-23T16:26:29Z)
• Identifiable Latent Polynomial Causal Models Through the Lens of Change [82.14087963690561]
  Causal representation learning aims to unveil latent high-level causal representations from observed low-level data. One of its primary tasks is to provide reliable assurance of identifying these latent causal models, known as identifiability.
  arXiv  Detail & Related papers  (2023-10-24T07:46:10Z)
• Improving the Reliability of Large Language Models by Leveraging Uncertainty-Aware In-Context Learning [76.98542249776257]
  Large-scale language models often face the challenge of "hallucination" We introduce an uncertainty-aware in-context learning framework to empower the model to enhance or reject its output in response to uncertainty.
  arXiv  Detail & Related papers  (2023-10-07T12:06:53Z)
• Learning for Counterfactual Fairness from Observational Data [62.43249746968616]
  Fairness-aware machine learning aims to eliminate biases of learning models against certain subgroups described by certain protected (sensitive) attributes such as race, gender, and age. A prerequisite for existing methods to achieve counterfactual fairness is the prior human knowledge of the causal model for the data. In this work, we address the problem of counterfactually fair prediction from observational data without given causal models by proposing a novel framework CLAIRE.
  arXiv  Detail & Related papers  (2023-07-17T04:08:29Z)
• Study of Distractors in Neural Models of Code [4.043200001974071]
  Finding important features that contribute to the prediction of neural models is an active area of research in explainable AI. In this work, we present an inverse perspective of distractor features: features that cast doubt about the prediction by affecting the model's confidence in its prediction. Our experiments across various tasks, models, and datasets of code reveal that the removal of tokens can have a significant impact on the confidence of models in their predictions.
  arXiv  Detail & Related papers  (2023-03-03T06:54:01Z)
• Fairness Increases Adversarial Vulnerability [50.90773979394264]
  This paper shows the existence of a dichotomy between fairness and robustness, and analyzes when achieving fairness decreases the model robustness to adversarial samples. Experiments on non-linear models and different architectures validate the theoretical findings in multiple vision domains. The paper proposes a simple, yet effective, solution to construct models achieving good tradeoffs between fairness and robustness.
  arXiv  Detail & Related papers  (2022-11-21T19:55:35Z)
• Exploring the Trade-off between Plausibility, Change Intensity and Adversarial Power in Counterfactual Explanations using Multi-objective Optimization [73.89239820192894]
  We argue that automated counterfactual generation should regard several aspects of the produced adversarial instances. We present a novel framework for the generation of counterfactual examples.
  arXiv  Detail & Related papers  (2022-05-20T15:02:53Z)
• Robustness in Deep Learning for Computer Vision: Mind the gap? [13.576376492050185]
  We identify, analyze, and summarize current definitions and progress towards non-adversarial robustness in deep learning for computer vision. We find that this area of research has received disproportionately little attention relative to adversarial machine learning.
  arXiv  Detail & Related papers  (2021-12-01T16:42:38Z)
• Learning Uncertainty with Artificial Neural Networks for Improved Remaining Time Prediction of Business Processes [0.15229257192293202]
  This paper is the first to apply these techniques to predictive process monitoring. We found that they contribute towards more accurate predictions and work quickly. This leads to many interesting applications, enables an earlier adoption of prediction systems with smaller datasets and fosters a better cooperation with humans.
  arXiv  Detail & Related papers  (2021-05-12T10:18:57Z)
• Accurate and Robust Feature Importance Estimation under Distribution Shifts [49.58991359544005]
  PRoFILE is a novel feature importance estimation method. We show significant improvements over state-of-the-art approaches, both in terms of fidelity and robustness.
  arXiv  Detail & Related papers  (2020-09-30T05:29:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.