Harmonizing the object recognition strategies of deep neural networks with humans

• URL: http://arxiv.org/abs/2211.04533v1
• Date: Tue, 8 Nov 2022 20:03:49 GMT
• Title: Harmonizing the object recognition strategies of deep neural networks with humans
• Authors: Thomas Fel, Ivan Felipe, Drew Linsley, Thomas Serre
• Abstract summary: We show that state-of-the-art deep neural networks (DNNs) are becoming less aligned with humans as their accuracy improves. Our work represents the first demonstration that the scaling laws that are guiding the design of DNNs today have also produced worse models of human vision.
• Score: 10.495114898741205
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The many successes of deep neural networks (DNNs) over the past decade have largely been driven by computational scale rather than insights from biological intelligence. Here, we explore if these trends have also carried concomitant improvements in explaining the visual strategies humans rely on for object recognition. We do this by comparing two related but distinct properties of visual strategies in humans and DNNs: where they believe important visual features are in images and how they use those features to categorize objects. Across 84 different DNNs trained on ImageNet and three independent datasets measuring the where and the how of human visual strategies for object recognition on those images, we find a systematic trade-off between DNN categorization accuracy and alignment with human visual strategies for object recognition. State-of-the-art DNNs are progressively becoming less aligned with humans as their accuracy improves. We rectify this growing issue with our neural harmonizer: a general-purpose training routine that both aligns DNN and human visual strategies and improves categorization accuracy. Our work represents the first demonstration that the scaling laws that are guiding the design of DNNs today have also produced worse models of human vision. We release our code and data at https://serre-lab.github.io/Harmonization to help the field build more human-like DNNs.

Related papers

• Dimensions underlying the representational alignment of deep neural networks with humans [3.1668470116181817]
  We propose a generic framework for yielding comparable representations in humans and deep neural networks (DNNs) Applying this framework to humans and a DNN model of natural images revealed a low-dimensional DNN embedding of both visual and semantic dimensions. In contrast to humans, DNNs exhibited a clear dominance of visual over semantic features, indicating divergent strategies for representing images.
  arXiv  Detail & Related papers  (2024-06-27T11:14:14Z)
• Graph Neural Networks for Brain Graph Learning: A Survey [53.74244221027981]
  Graph neural networks (GNNs) have demonstrated a significant advantage in mining graph-structured data. GNNs to learn brain graph representations for brain disorder analysis has recently gained increasing attention. In this paper, we aim to bridge this gap by reviewing brain graph learning works that utilize GNNs.
  arXiv  Detail & Related papers  (2024-06-01T02:47:39Z)
• Fixing the problems of deep neural networks will require better training data and learning algorithms [20.414456664907316]
  We argue that DNNs are poor models of biological vision because they rely on strategies that differ markedly from those of humans. We show that this problem is worsening as DNNs are becoming larger-scale and increasingly more accurate.
  arXiv  Detail & Related papers  (2023-09-26T03:09:00Z)
• Performance-optimized deep neural networks are evolving into worse models of inferotemporal visual cortex [8.45100792118802]
  We show that object recognition accuracy of deep neural networks (DNNs) correlates with their ability to predict neural responses to natural images in the inferotemporal (IT) cortex. Our results suggest that harmonized DNNs break the trade-off between ImageNet accuracy and neural prediction accuracy.
  arXiv  Detail & Related papers  (2023-06-06T15:34:45Z)
• Adversarial alignment: Breaking the trade-off between the strength of an attack and its relevance to human perception [10.883174135300418]
  Adversarial attacks have long been considered the "Achilles' heel" of deep learning. Here, we investigate how the robustness of DNNs to adversarial attacks has evolved as their accuracy on ImageNet has continued to improve.
  arXiv  Detail & Related papers  (2023-06-05T20:26:17Z)
• Are Deep Neural Networks Adequate Behavioural Models of Human Visual Perception? [8.370048099732573]
  Deep neural networks (DNNs) are machine learning algorithms that have revolutionised computer vision. We argue that it is important to distinguish between statistical tools and computational models. We dispel a number of myths surrounding DNNs in vision science.
  arXiv  Detail & Related papers  (2023-05-26T15:31:06Z)
• Deep Reinforcement Learning Guided Graph Neural Networks for Brain Network Analysis [61.53545734991802]
  We propose a novel brain network representation framework, namely BN-GNN, which searches for the optimal GNN architecture for each brain network. Our proposed BN-GNN improves the performance of traditional GNNs on different brain network analysis tasks.
  arXiv  Detail & Related papers  (2022-03-18T07:05:27Z)
• Overcoming the Domain Gap in Neural Action Representations [60.47807856873544]
  3D pose data can now be reliably extracted from multi-view video sequences without manual intervention. We propose to use it to guide the encoding of neural action representations together with a set of neural and behavioral augmentations. To reduce the domain gap, during training, we swap neural and behavioral data across animals that seem to be performing similar actions.
  arXiv  Detail & Related papers  (2021-12-02T12:45:46Z)
• How Do Adam and Training Strategies Help BNNs Optimization? [50.22482900678071]
  We show that Adam is better equipped to handle the rugged loss surface of BNNs and reaches a better optimum with higher generalization ability. We derive a simple training scheme, building on existing Adam-based optimization, which achieves 70.5% top-1 accuracy on the ImageNet dataset.
  arXiv  Detail & Related papers  (2021-06-21T17:59:51Z)
• Continuous Emotion Recognition with Spatiotemporal Convolutional Neural Networks [82.54695985117783]
  We investigate the suitability of state-of-the-art deep learning architectures for continuous emotion recognition using long video sequences captured in-the-wild. We have developed and evaluated convolutional recurrent neural networks combining 2D-CNNs and long short term-memory units, and inflated 3D-CNN models, which are built by inflating the weights of a pre-trained 2D-CNN model during fine-tuning.
  arXiv  Detail & Related papers  (2020-11-18T13:42:05Z)
• Boosting Deep Neural Networks with Geometrical Prior Knowledge: A Survey [77.99182201815763]
  Deep Neural Networks (DNNs) achieve state-of-the-art results in many different problem settings. DNNs are often treated as black box systems, which complicates their evaluation and validation. One promising field, inspired by the success of convolutional neural networks (CNNs) in computer vision tasks, is to incorporate knowledge about symmetric geometrical transformations.
  arXiv  Detail & Related papers  (2020-06-30T14:56:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.