Do Vision Encoders Exhibit Human-like Color Thresholds?

• URL: http://arxiv.org/abs/2607.16540v1
• Date: Fri, 17 Jul 2026 22:30:12 GMT
• Title: Do Vision Encoders Exhibit Human-like Color Thresholds?
• Abstract summary: In recent years, deep neural networks have become the standard networks for computer vision tasks.<n>Despite the widespread use of deep vision encoders, few studies have investigated whether their internal representations exhibit human-like discrimination thresholds.<n>We present a large-scale exploratory study probing the chromatic sensitivity of more than 50 pretrained vision encoders.
• Score: 44.97183276701248
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Understanding and characterizing human color perception is a longstanding research goal. One of the most traditional approaches is looking for the human color discrimination thresholds, the minimum chromatic differences perceptible to human observers. In recent years, deep neural networks have become the standard networks for computer vision tasks. In particular, deep vision encoders, foundation models trained on large-scale visual data, map images into latent feature representations. Despite the widespread use of deep vision encoders, few studies have investigated whether their internal representations exhibit human-like discrimination thresholds. In this work, we present a large-scale exploratory study probing the chromatic sensitivity of more than 50 pretrained vision encoders, including convolutional networks and vision transformers, against human discrimination thresholds. Using controlled chromatic stimuli at multiple chroma levels, we compare model-derived chromatic discrimination thresholds with human discrimination ellipses through a region-overlap metric (mIoU). Our analysis reveals generally weak alignment between model representations and human perceptual thresholds across all model families, with the best mIoU < 0.25. Moreover, we find that self-supervised encoders consistently outperform supervised ones, while language-supervised models show the most polarized behavior, occupying both the top and bottom of the ranking. These findings suggest that human-like chromatic sensitivity does not emerge naturally from current large-scale visual training objectives for any of the analyzed architectures.

Related papers

• Human-AI Perceptual Alignment by Playing Hues and Cues [2.6369756143132634]
  evaluating Contrastive Vision-Language Models (CVLMs) and humans is typically constrained by traditional benchmarks that overlook fine-grained semantic and cultural nuances.<n>We propose a novel evaluation framework that leverages the gamified, discrete color space of the board game Hues and Cues.<n>We evaluate 162 models across multiple architectural families and pre-training datasets to assess their semantic color grounding.
  arXiv  Detail & Related papers  (2026-08-07T12:00:55Z)
• An extremely coarse feedback signal is sufficient for learning human-aligned visual representations [2.9506547907696006]
  We investigate how the coarseness of a learning signal shapes representational alignment with human vision.<n>We train hundreds of neural networks across convolutional and transformer architectures on these coarse classification tasks.<n>We find that networks trained to distinguish as few as 8 broad categories learn representations that match or exceed the neural alignment of models distinguishing 1,000-classes.
  arXiv  Detail & Related papers  (2026-05-07T01:02:21Z)
• Human Cognitive Benchmarks Reveal Foundational Visual Gaps in MLLMs [65.93003087656754]
  VisFactor is a benchmark that digitizes 20 vision-centric subtests from a well-established cognitive psychology assessment.<n>We evaluate 20 frontier Multimodal Large Language Models (MLLMs) from GPT, Gemini, Claude, LLaMA, Qwen, and SEED families.<n>The best-performing model achieves a score of only 25.19 out of 100, with consistent failures on tasks such as mental rotation, spatial relation inference, and figure-ground discrimination.
  arXiv  Detail & Related papers  (2025-02-23T04:21:32Z)
• When Does Perceptual Alignment Benefit Vision Representations? [76.32336818860965]
  We investigate how aligning vision model representations to human perceptual judgments impacts their usability. We find that aligning models to perceptual judgments yields representations that improve upon the original backbones across many downstream tasks. Our results suggest that injecting an inductive bias about human perceptual knowledge into vision models can contribute to better representations.
  arXiv  Detail & Related papers  (2024-10-14T17:59:58Z)
• ColorSense: A Study on Color Vision in Machine Visual Recognition [57.916512479603064]
  We collect 110,000 non-trivial human annotations of foreground and background color labels from visual recognition benchmarks.<n>We validate the use of our datasets by demonstrating that the level of color discrimination has a dominating effect on the performance of machine perception models.<n>Our findings suggest that object recognition tasks such as classification and localization are susceptible to color vision bias.
  arXiv  Detail & Related papers  (2022-12-16T18:51:41Z)
• A domain adaptive deep learning solution for scanpath prediction of paintings [66.46953851227454]
  This paper focuses on the eye-movement analysis of viewers during the visual experience of a certain number of paintings. We introduce a new approach to predicting human visual attention, which impacts several cognitive functions for humans. The proposed new architecture ingests images and returns scanpaths, a sequence of points featuring a high likelihood of catching viewers' attention.
  arXiv  Detail & Related papers  (2022-09-22T22:27:08Z)
• Guiding Visual Attention in Deep Convolutional Neural Networks Based on Human Eye Movements [0.0]
  Deep Convolutional Neural Networks (DCNNs) were originally inspired by principles of biological vision. Recent advances in deep learning seem to decrease this similarity. We investigate a purely data-driven approach to obtain useful models.
  arXiv  Detail & Related papers  (2022-06-21T17:59:23Z)
• Are Convolutional Neural Networks or Transformers more like human vision? [9.83454308668432]
  We show that attention-based networks can achieve higher accuracy than CNNs on vision tasks. These results have implications both for building more human-like vision models, as well as for understanding visual object recognition in humans.
  arXiv  Detail & Related papers  (2021-05-15T10:33:35Z)
• Appearance Consensus Driven Self-Supervised Human Mesh Recovery [67.20942777949793]
  We present a self-supervised human mesh recovery framework to infer human pose and shape from monocular images. We achieve state-of-the-art results on the standard model-based 3D pose estimation benchmarks. The resulting colored mesh prediction opens up the usage of our framework for a variety of appearance-related tasks beyond the pose and shape estimation.
  arXiv  Detail & Related papers  (2020-08-04T05:40:39Z)
• Seeing eye-to-eye? A comparison of object recognition performance in humans and deep convolutional neural networks under image manipulation [0.0]
  This study aims towards a behavioral comparison of visual core object recognition performance between humans and feedforward neural networks. Analyses of accuracy revealed that humans not only outperform DCNNs on all conditions, but also display significantly greater robustness towards shape and most notably color alterations.
  arXiv  Detail & Related papers  (2020-07-13T10:26:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.