Perceptual misalignment of texture representations in convolutional neural networks

• URL: http://arxiv.org/abs/2604.01341v1
• Date: Wed, 01 Apr 2026 19:51:45 GMT
• Title: Perceptual misalignment of texture representations in convolutional neural networks
• Abstract summary: We compare the perceptual content captured by feature correlations computed for a diverse pool of convolutional neural networks (CNNs)<n>We find no connection between conventional measures of CNN quality as a model of the visual system and its alignment with human texture perception.<n>We conclude that texture perception involves mechanisms that are distinct from those that are commonly modeled using approaches based on CNNs trained on object recognition.
• Score: 3.68986335765876
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Mathematical modeling of visual textures traces back to Julesz's intuition that texture perception in humans is based on local correlations between image features. An influential approach for texture analysis and generation generalizes this notion to linear correlations between the nonlinear features computed by convolutional neural networks (CNNs), compiled into Gram matrices. Given that CNNs are often used as models for the visual system, it is natural to ask whether such "texture representations" spontaneously align with the textures' perceptual content, and in particular whether those CNNs that are regarded as better models for the visual system also possess more human-like texture representations. Here we compare the perceptual content captured by feature correlations computed for a diverse pool of CNNs, and we compare it to the models' perceptual alignment with the mammalian visual system as measured by Brain-Score. Surprisingly, we find that there is no connection between conventional measures of CNN quality as a model of the visual system and its alignment with human texture perception. We conclude that texture perception involves mechanisms that are distinct from those that are commonly modeled using approaches based on CNNs trained on object recognition, possibly depending on the integration of contextual information.

Related papers

• Texture Representations in Deep Vision Models: Comparing CNNs, Vision Transformers, and Human Perception [45.62648262593701]
  This study focuses on another aspect of vision: texture perception.<n>We consider textures of different complexity generated with three different algorithms from the same source images.<n>We find that the representation of textures is aligned in different ViTs, but not between the ViTs and the CNN.
  arXiv  Detail & Related papers  (2026-07-09T10:05:43Z)
• ImageNet-trained CNNs are not biased towards texture: Revisiting feature reliance through controlled suppression [10.20881592893918]
  We revisit the hypothesis that Convolutional Neural Networks (CNNs) are inherently texture-biased.<n>We propose a domain-agnostic framework that quantifies feature reliance through systematic suppression of shape, texture, and color cues.<n>We extend the analysis across computer vision, medical imaging, and remote sensing, revealing that reliance patterns differ systematically.
  arXiv  Detail & Related papers  (2025-09-24T15:24:43Z)
• Graph Neural Networks for Learning Equivariant Representations of Neural Networks [55.04145324152541]
  We propose to represent neural networks as computational graphs of parameters. Our approach enables a single model to encode neural computational graphs with diverse architectures. We showcase the effectiveness of our method on a wide range of tasks, including classification and editing of implicit neural representations.
  arXiv  Detail & Related papers  (2024-03-18T18:01:01Z)
• Connecting metrics for shape-texture knowledge in computer vision [1.7785095623975342]
  Deep neural networks remain brittle and susceptible to many changes in the image that do not cause humans to misclassify images. Part of this different behavior may be explained by the type of features humans and deep neural networks use in vision tasks.
  arXiv  Detail & Related papers  (2023-01-25T14:37:42Z)
• DeepDC: Deep Distance Correlation as a Perceptual Image Quality Evaluator [53.57431705309919]
  ImageNet pre-trained deep neural networks (DNNs) show notable transferability for building effective image quality assessment (IQA) models. We develop a novel full-reference IQA (FR-IQA) model based exclusively on pre-trained DNN features. We conduct comprehensive experiments to demonstrate the superiority of the proposed quality model on five standard IQA datasets.
  arXiv  Detail & Related papers  (2022-11-09T14:57:27Z)
• Prune and distill: similar reformatting of image information along rat visual cortex and deep neural networks [61.60177890353585]
  Deep convolutional neural networks (CNNs) have been shown to provide excellent models for its functional analogue in the brain, the ventral stream in visual cortex. Here we consider some prominent statistical patterns that are known to exist in the internal representations of either CNNs or the visual cortex. We show that CNNs and visual cortex share a similarly tight relationship between dimensionality expansion/reduction of object representations and reformatting of image information.
  arXiv  Detail & Related papers  (2022-05-27T08:06:40Z)
• Informative Dropout for Robust Representation Learning: A Shape-bias Perspective [84.30946377024297]
  We propose a light-weight model-agnostic method, namely Informative Dropout (InfoDrop), to improve interpretability and reduce texture bias. Specifically, we discriminate texture from shape based on local self-information in an image, and adopt a Dropout-like algorithm to decorrelate the model output from the local texture.
  arXiv  Detail & Related papers  (2020-08-10T16:52:24Z)
• Teaching CNNs to mimic Human Visual Cognitive Process & regularise Texture-Shape bias [18.003188982585737]
  Recent experiments in computer vision demonstrate texture bias as the primary reason for supreme results in models employing Convolutional Neural Networks (CNNs) It is believed that the cost function forces the CNN to take a greedy approach and develop a proclivity for local information like texture to increase accuracy, thus failing to explore any global statistics. We propose CognitiveCNN, a new intuitive architecture, inspired from feature integration theory in psychology to utilise human interpretable feature like shape, texture, edges etc. to reconstruct, and classify the image.
  arXiv  Detail & Related papers  (2020-06-25T22:32:54Z)
• Texture Interpolation for Probing Visual Perception [4.637185817866918]
  We show that distributions of deep convolutional neural network (CNN) activations of a texture are well described by elliptical distributions. We then propose the natural geodesics arising with the optimal transport metric to interpolate between arbitrary textures. Compared to other CNN-based approaches, our method appears to match more closely the geometry of texture perception.
  arXiv  Detail & Related papers  (2020-06-05T21:28:36Z)
• Neural Human Video Rendering by Learning Dynamic Textures and Rendering-to-Video Translation [99.64565200170897]
  We propose a novel human video synthesis method by explicitly disentangling the learning of time-coherent fine-scale details from the embedding of the human in 2D screen space. We show several applications of our approach, such as human reenactment and novel view synthesis from monocular video, where we show significant improvement over the state of the art both qualitatively and quantitatively.
  arXiv  Detail & Related papers  (2020-01-14T18:06:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.