Perceptual-Score: A Psychophysical Measure for Assessing the Biological Plausibility of Visual Recognition Models

• URL: http://arxiv.org/abs/2210.08632v1
• Date: Sun, 16 Oct 2022 20:34:26 GMT
• Title: Perceptual-Score: A Psychophysical Measure for Assessing the Biological Plausibility of Visual Recognition Models
• Authors: Brandon RichardWebster, Anthony DiFalco, Elisabetta Caldesi, Walter J. Scheirer
• Abstract summary: This article proposes a new metric, Perceptual-Score, which is grounded in visual psychophysics. We perform the procedure on twelve models that vary in degree of biological inspiration and complexity. Each model's Perceptual-Score is compared against the state-of-the-art neural activity-based metric, Brain-Score.
• Score: 9.902669518047714
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: For the last decade, convolutional neural networks (CNNs) have vastly superseded their predecessors in nearly all vision tasks in artificial intelligence, including object recognition. However, in spite of abundant advancements, they continue to pale in comparison to biological vision. This chasm has prompted the development of biologically-inspired models that have attempted to mimic the human visual system, primarily at a neural-level, which are evaluated using standard dataset benchmarks. However, more work is needed to understand how these models actually perceive the visual world. This article proposes a state-of-the-art procedure that generates a new metric, Perceptual-Score, which is grounded in visual psychophysics, and is capable of reliably estimating perceptual responses across numerous models -- representing a large range in complexity and biological inspiration. We perform the procedure on twelve models that vary in degree of biological inspiration and complexity, and compare the results against the aggregated results of 2,390 Amazon Mechanical Turk workers who together provided ~2.7 million perceptual responses. Each model's Perceptual-Score is compared against the state-of-the-art neural activity-based metric, Brain-Score. Our study indicates that models with high correlation to human perceptual behavior also have high correlation with the corresponding neural activity.

Related papers

• Neural Dynamics Model of Visual Decision-Making: Learning from Human Experts [28.340344705437758]
  We implement a comprehensive visual decision-making model that spans from visual input to behavioral output. Our model aligns closely with human behavior and reflects neural activities in primates. A neuroimaging-informed fine-tuning approach was introduced and applied to the model, leading to performance improvements.
  arXiv  Detail & Related papers  (2024-09-04T02:38:52Z)
• Brain-inspired Computational Modeling of Action Recognition with Recurrent Spiking Neural Networks Equipped with Reinforcement Delay Learning [4.9798155883849935]
  Action recognition has received significant attention due to its intricate nature and the brain's exceptional performance in this area. Current solutions for action recognition either exhibit limitations in effectively addressing the problem or lack the necessary biological plausibility. This article presents an effective brain-inspired computational model for action recognition.
  arXiv  Detail & Related papers  (2024-06-17T17:34:16Z)
• Modelling Human Visual Motion Processing with Trainable Motion Energy Sensing and a Self-attention Network [1.9458156037869137]
  We propose an image-computable model of human motion perception by bridging the gap between biological and computer vision models. This model architecture aims to capture the computations in V1-MT, the core structure for motion perception in the biological visual system. In silico neurophysiology reveals that our model's unit responses are similar to mammalian neural recordings regarding motion pooling and speed tuning.
  arXiv  Detail & Related papers  (2023-05-16T04:16:07Z)
• Guiding Visual Attention in Deep Convolutional Neural Networks Based on Human Eye Movements [0.0]
  Deep Convolutional Neural Networks (DCNNs) were originally inspired by principles of biological vision. Recent advances in deep learning seem to decrease this similarity. We investigate a purely data-driven approach to obtain useful models.
  arXiv  Detail & Related papers  (2022-06-21T17:59:23Z)
• Overcoming the Domain Gap in Neural Action Representations [60.47807856873544]
  3D pose data can now be reliably extracted from multi-view video sequences without manual intervention. We propose to use it to guide the encoding of neural action representations together with a set of neural and behavioral augmentations. To reduce the domain gap, during training, we swap neural and behavioral data across animals that seem to be performing similar actions.
  arXiv  Detail & Related papers  (2021-12-02T12:45:46Z)
• Overcoming the Domain Gap in Contrastive Learning of Neural Action Representations [60.47807856873544]
  A fundamental goal in neuroscience is to understand the relationship between neural activity and behavior. We generated a new multimodal dataset consisting of the spontaneous behaviors generated by fruit flies. This dataset and our new set of augmentations promise to accelerate the application of self-supervised learning methods in neuroscience.
  arXiv  Detail & Related papers  (2021-11-29T15:27:51Z)
• Towards a Predictive Processing Implementation of the Common Model of Cognition [79.63867412771461]
  We describe an implementation of the common model of cognition grounded in neural generative coding and holographic associative memory. The proposed system creates the groundwork for developing agents that learn continually from diverse tasks as well as model human performance at larger scales.
  arXiv  Detail & Related papers  (2021-05-15T22:55:23Z)
• The Neural Coding Framework for Learning Generative Models [91.0357317238509]
  We propose a novel neural generative model inspired by the theory of predictive processing in the brain. In a similar way, artificial neurons in our generative model predict what neighboring neurons will do, and adjust their parameters based on how well the predictions matched reality.
  arXiv  Detail & Related papers  (2020-12-07T01:20:38Z)
• Continuous Emotion Recognition with Spatiotemporal Convolutional Neural Networks [82.54695985117783]
  We investigate the suitability of state-of-the-art deep learning architectures for continuous emotion recognition using long video sequences captured in-the-wild. We have developed and evaluated convolutional recurrent neural networks combining 2D-CNNs and long short term-memory units, and inflated 3D-CNN models, which are built by inflating the weights of a pre-trained 2D-CNN model during fine-tuning.
  arXiv  Detail & Related papers  (2020-11-18T13:42:05Z)
• Fooling the primate brain with minimal, targeted image manipulation [67.78919304747498]
  We propose an array of methods for creating minimal, targeted image perturbations that lead to changes in both neuronal activity and perception as reflected in behavior. Our work shares the same goal with adversarial attack, namely the manipulation of images with minimal, targeted noise that leads ANN models to misclassify the images.
  arXiv  Detail & Related papers  (2020-11-11T08:30:54Z)
• Seeing eye-to-eye? A comparison of object recognition performance in humans and deep convolutional neural networks under image manipulation [0.0]
  This study aims towards a behavioral comparison of visual core object recognition performance between humans and feedforward neural networks. Analyses of accuracy revealed that humans not only outperform DCNNs on all conditions, but also display significantly greater robustness towards shape and most notably color alterations.
  arXiv  Detail & Related papers  (2020-07-13T10:26:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.