On Human Visual Contrast Sensitivity and Machine Vision Robustness: A
Comparative Study
- URL: http://arxiv.org/abs/2212.08650v1
- Date: Fri, 16 Dec 2022 18:51:41 GMT
- Title: On Human Visual Contrast Sensitivity and Machine Vision Robustness: A
Comparative Study
- Authors: Ming-Chang Chiu, Yingfei Wang, Derrick Eui Gyu Kim, Pin-Yu Chen,
Xuezhe Ma
- Abstract summary: How color differences affect machine vision has not been well explored.
Our work tries to bridge this gap between the human color vision aspect of visual recognition and that of the machine.
We devise a new framework in two dimensions to perform extensive analyses on the effect of color contrast and corrupted images.
- Score: 68.41864523774164
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: It is well established in neuroscience that color vision plays an essential
part in the human visual perception system. Meanwhile, many novel designs for
computer vision inspired by human vision have achieved success in a wide range
of tasks and applications. Nonetheless, how color differences affect machine
vision has not been well explored. Our work tries to bridge this gap between
the human color vision aspect of visual recognition and that of the machine. To
achieve this, we curate two datasets: CIFAR10-F and CIFAR100-F, which are based
on the foreground colors of the popular CIFAR datasets. Together with CIFAR10-B
and CIFAR100-B, the existing counterpart datasets with information on the
background colors of CIFAR test sets, we assign each image based on its color
contrast level per its foreground and background color labels and use this as a
proxy to study how color contrast affects machine vision. We first conduct a
proof-of-concept study, showing the effect of color difference and validate our
datasets. Furthermore, on a broader level, an important characteristic of human
vision is its robustness against ambient changes; therefore, drawing
inspirations from ophthalmology and the robustness literature, we analogize
contrast sensitivity from the human visual aspect to machine vision and
complement the current robustness study using corrupted images with our
CIFAR-CoCo datasets. In summary, motivated by neuroscience and equipped with
the datasets we curate, we devise a new framework in two dimensions to perform
extensive analyses on the effect of color contrast and corrupted images: (1)
model architecture, (2) model size, to measure the perception ability of
machine vision beyond total accuracy. We also explore how task complexity and
data augmentation play a role in this setup. Our results call attention to new
evaluation approaches for human-like machine perception.
Related papers
- For a semiotic AI: Bridging computer vision and visual semiotics for computational observation of large scale facial image archives [3.418398936676879]
This work presents FRESCO, a framework designed to explore the socio-cultural implications of images on social media platforms at scale.
FRESCO deconstructs images into numerical and categorical variables using state-of-the-art computer vision techniques.
The framework analyzes images across three levels: the plastic level, encompassing fundamental visual features like lines and colors; the figurative level, representing specific entities or concepts; and the enunciation level, which focuses particularly on constructing the point of view of the spectator and observer.
arXiv Detail & Related papers (2024-07-03T16:57:38Z) - Freeview Sketching: View-Aware Fine-Grained Sketch-Based Image Retrieval [85.73149096516543]
We address the choice of viewpoint during sketch creation in Fine-Grained Sketch-Based Image Retrieval (FG-SBIR)
A pilot study highlights the system's struggle when query-sketches differ in viewpoint from target instances.
To reconcile this, we advocate for a view-aware system, seamlessly accommodating both view-agnostic and view-specific tasks.
arXiv Detail & Related papers (2024-07-01T21:20:44Z) - Comparative Analysis Of Color Models For Human Perception And Visual Color Difference [0.0]
The study evaluates color models such as RGB, HSV, HSL, XYZ, CIELAB, and CIELUV to assess their effectiveness in accurately representing how humans perceive color.
In image processing, accurate assessment of color difference is essential for applications ranging from digital design to quality control.
arXiv Detail & Related papers (2024-06-27T20:41:49Z) - Divergences in Color Perception between Deep Neural Networks and Humans [3.0315685825606633]
We develop experiments for evaluating the perceptual coherence of color embeddings in deep neural networks (DNNs)
We assess how well these algorithms predict human color similarity judgments collected via an online survey.
We compare DNN performance against an interpretable and cognitively plausible model of color perception based on wavelet decomposition.
arXiv Detail & Related papers (2023-09-11T20:26:40Z) - Edge-Aware Image Color Appearance and Difference Modeling [0.0]
Humans have developed a keen sense of color and are able to detect subtle differences in appearance.
Applying contrast sensitivity functions and local adaptation rules in an edge-aware manner improves image difference predictions.
arXiv Detail & Related papers (2023-04-20T22:55:16Z) - The Influences of Color and Shape Features in Visual Contrastive
Learning [0.0]
This paper investigates the influences of individual image features (e.g., color and shape) to model performance remain ambiguous.
Experimental results show that compared with supervised representations, contrastive representations tend to cluster with objects of similar color.
arXiv Detail & Related papers (2023-01-29T15:10:14Z) - A domain adaptive deep learning solution for scanpath prediction of
paintings [66.46953851227454]
This paper focuses on the eye-movement analysis of viewers during the visual experience of a certain number of paintings.
We introduce a new approach to predicting human visual attention, which impacts several cognitive functions for humans.
The proposed new architecture ingests images and returns scanpaths, a sequence of points featuring a high likelihood of catching viewers' attention.
arXiv Detail & Related papers (2022-09-22T22:27:08Z) - Peripheral Vision Transformer [52.55309200601883]
We take a biologically inspired approach and explore to model peripheral vision in deep neural networks for visual recognition.
We propose to incorporate peripheral position encoding to the multi-head self-attention layers to let the network learn to partition the visual field into diverse peripheral regions given training data.
We evaluate the proposed network, dubbed PerViT, on the large-scale ImageNet dataset and systematically investigate the inner workings of the model for machine perception.
arXiv Detail & Related papers (2022-06-14T12:47:47Z) - What Can You Learn from Your Muscles? Learning Visual Representation
from Human Interactions [50.435861435121915]
We use human interaction and attention cues to investigate whether we can learn better representations compared to visual-only representations.
Our experiments show that our "muscly-supervised" representation outperforms a visual-only state-of-the-art method MoCo.
arXiv Detail & Related papers (2020-10-16T17:46:53Z) - Appearance Consensus Driven Self-Supervised Human Mesh Recovery [67.20942777949793]
We present a self-supervised human mesh recovery framework to infer human pose and shape from monocular images.
We achieve state-of-the-art results on the standard model-based 3D pose estimation benchmarks.
The resulting colored mesh prediction opens up the usage of our framework for a variety of appearance-related tasks beyond the pose and shape estimation.
arXiv Detail & Related papers (2020-08-04T05:40:39Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.