Evaluating Model Perception of Color Illusions in Photorealistic Scenes

• URL: http://arxiv.org/abs/2412.06184v1
• Date: Mon, 09 Dec 2024 03:49:10 GMT
• Title: Evaluating Model Perception of Color Illusions in Photorealistic Scenes
• Authors: Lingjun Mao, Zineng Tang, Alane Suhr,
• Abstract summary: We study the perception of color illusions by vision-language models. We propose an automated framework for generating color illusion images. Experiments show that all studied VLMs exhibit perceptual biases similar human vision.
• Score: 16.421832484760987
• License:
• Abstract: We study the perception of color illusions by vision-language models. Color illusion, where a person's visual system perceives color differently from actual color, is well-studied in human vision. However, it remains underexplored whether vision-language models (VLMs), trained on large-scale human data, exhibit similar perceptual biases when confronted with such color illusions. We propose an automated framework for generating color illusion images, resulting in RCID (Realistic Color Illusion Dataset), a dataset of 19,000 realistic illusion images. Our experiments show that all studied VLMs exhibit perceptual biases similar human vision. Finally, we train a model to distinguish both human perception and actual pixel differences.

Related papers

• IllusionBench: A Large-scale and Comprehensive Benchmark for Visual Illusion Understanding in Vision-Language Models [56.34742191010987]
  Current Visual Language Models (VLMs) show impressive image understanding but struggle with visual illusions. We introduce IllusionBench, a comprehensive visual illusion dataset that encompasses classic cognitive illusions and real-world scene illusions. We design trap illusions that resemble classical patterns but differ in reality, highlighting issues in SOTA models.
  arXiv  Detail & Related papers  (2025-01-01T14:10:25Z)
• The Art of Deception: Color Visual Illusions and Diffusion Models [55.830105086695]
  Recent studies have shown that artificial neural networks (ANNs) can also be deceived by visual illusions. We show how visual illusions are encoded in diffusion models. We also show how to generate new unseen visual illusions in realistic images using text-to-image diffusion models.
  arXiv  Detail & Related papers  (2024-12-13T13:07:08Z)
• When Does Perceptual Alignment Benefit Vision Representations? [76.32336818860965]
  We investigate how aligning vision model representations to human perceptual judgments impacts their usability. We find that aligning models to perceptual judgments yields representations that improve upon the original backbones across many downstream tasks. Our results suggest that injecting an inductive bias about human perceptual knowledge into vision models can contribute to better representations.
  arXiv  Detail & Related papers  (2024-10-14T17:59:58Z)
• Evaluating Multiview Object Consistency in Humans and Image Models [68.36073530804296]
  We leverage an experimental design from the cognitive sciences which requires zero-shot visual inferences about object shape. We collect 35K trials of behavioral data from over 500 participants. We then evaluate the performance of common vision models.
  arXiv  Detail & Related papers  (2024-09-09T17:59:13Z)
• Comparative Analysis Of Color Models For Human Perception And Visual Color Difference [0.0]
  The study evaluates color models such as RGB, HSV, HSL, XYZ, CIELAB, and CIELUV to assess their effectiveness in accurately representing how humans perceive color. In image processing, accurate assessment of color difference is essential for applications ranging from digital design to quality control.
  arXiv  Detail & Related papers  (2024-06-27T20:41:49Z)
• Investigating Color Illusions from the Perspective of Computational Color Constancy [2.608935407927351]
  We argue that any model that can reproduce our sensation on color illusions should also be able to provide pixel-wise estimates of the light source. In this study, we take several color constancy methods and modify them to reproduce the behavior of the human visual system on color illusions.
  arXiv  Detail & Related papers  (2023-12-20T15:34:15Z)
• Grounding Visual Illusions in Language: Do Vision-Language Models Perceive Illusions Like Humans? [28.654771227396807]
  Vision-Language Models (VLMs) are trained on vast amounts of data captured by humans emulating our understanding of the world. Do VLMs have the similar kind of illusions as humans do, or do they faithfully learn to represent reality? We build a dataset containing five types of visual illusions and formulate four tasks to examine visual illusions in state-of-the-art VLMs.
  arXiv  Detail & Related papers  (2023-10-31T18:01:11Z)
• Divergences in Color Perception between Deep Neural Networks and Humans [3.0315685825606633]
  We develop experiments for evaluating the perceptual coherence of color embeddings in deep neural networks (DNNs) We assess how well these algorithms predict human color similarity judgments collected via an online survey. We compare DNN performance against an interpretable and cognitively plausible model of color perception based on wavelet decomposition.
  arXiv  Detail & Related papers  (2023-09-11T20:26:40Z)
• ColorSense: A Study on Color Vision in Machine Visual Recognition [57.916512479603064]
  We collect 110,000 non-trivial human annotations of foreground and background color labels from visual recognition benchmarks. We validate the use of our datasets by demonstrating that the level of color discrimination has a dominating effect on the performance of machine perception models. Our findings suggest that object recognition tasks such as classification and localization are susceptible to color vision bias.
  arXiv  Detail & Related papers  (2022-12-16T18:51:41Z)
• Towards Photorealistic Colorization by Imagination [48.82757902812846]
  We present a novel approach to automatic image colorization by imitating the imagination process of human experts. Our work produces more colorful and diverse results than state-of-the-art image colorization methods.
  arXiv  Detail & Related papers  (2021-08-20T14:28:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.