Robustness of Humans and Machines on Object Recognition with Extreme Image Transformations

• URL: http://arxiv.org/abs/2205.05167v1
• Date: Mon, 9 May 2022 17:15:54 GMT
• Title: Robustness of Humans and Machines on Object Recognition with Extreme Image Transformations
• Authors: Dakarai Crowder and Girik Malik
• Abstract summary: We introduce a novel set of image transforms and evaluate humans and networks on an object recognition task. We found performance for a few common networks quickly decreases while humans are able to recognize objects with a high accuracy.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent neural network architectures have claimed to explain data from the human visual cortex. Their demonstrated performance is however still limited by the dependence on exploiting low-level features for solving visual tasks. This strategy limits their performance in case of out-of-distribution/adversarial data. Humans, meanwhile learn abstract concepts and are mostly unaffected by even extreme image distortions. Humans and networks employ strikingly different strategies to solve visual tasks. To probe this, we introduce a novel set of image transforms and evaluate humans and networks on an object recognition task. We found performance for a few common networks quickly decreases while humans are able to recognize objects with a high accuracy.

Related papers

• Saliency Suppressed, Semantics Surfaced: Visual Transformations in Neural Networks and the Brain [0.0]
  We take inspiration from neuroscience to shed light on how neural networks encode information at low (visual saliency) and high (semantic similarity) levels of abstraction. We find that ResNets are more sensitive to saliency information than ViTs, when trained with object classification objectives. We show that semantic encoding is a key factor in aligning AI with human visual perception, while saliency suppression is a non-brain-like strategy.
  arXiv  Detail & Related papers  (2024-04-29T15:05:42Z)
• Degraded Polygons Raise Fundamental Questions of Neural Network Perception [5.423100066629618]
  We revisit the task of recovering images under degradation, first introduced over 30 years ago in the Recognition-by-Components theory of human vision. We implement the Automated Shape Recoverability Test for rapidly generating large-scale datasets of perimeter-degraded regular polygons. We find that neural networks' behavior on this simple task conflicts with human behavior.
  arXiv  Detail & Related papers  (2023-06-08T06:02:39Z)
• Connecting metrics for shape-texture knowledge in computer vision [1.7785095623975342]
  Deep neural networks remain brittle and susceptible to many changes in the image that do not cause humans to misclassify images. Part of this different behavior may be explained by the type of features humans and deep neural networks use in vision tasks.
  arXiv  Detail & Related papers  (2023-01-25T14:37:42Z)
• Extreme Image Transformations Affect Humans and Machines Differently [0.0]
  Some recent artificial neural networks (ANNs) claim to model aspects of primate neural and human performance data. We introduce a set of novel image transforms inspired by neurophysiological findings and evaluate humans and ANNs on an object recognition task. We show that machines perform better than humans for certain transforms and struggle to perform at par with humans on others that are easy for humans.
  arXiv  Detail & Related papers  (2022-11-30T18:12:53Z)
• A domain adaptive deep learning solution for scanpath prediction of paintings [66.46953851227454]
  This paper focuses on the eye-movement analysis of viewers during the visual experience of a certain number of paintings. We introduce a new approach to predicting human visual attention, which impacts several cognitive functions for humans. The proposed new architecture ingests images and returns scanpaths, a sequence of points featuring a high likelihood of catching viewers' attention.
  arXiv  Detail & Related papers  (2022-09-22T22:27:08Z)
• FuNNscope: Visual microscope for interactively exploring the loss landscape of fully connected neural networks [77.34726150561087]
  We show how to explore high-dimensional landscape characteristics of neural networks. We generalize observations on small neural networks to more complex systems. An interactive dashboard opens up a number of possible application networks.
  arXiv  Detail & Related papers  (2022-04-09T16:41:53Z)
• Towards robust vision by multi-task learning on monkey visual cortex [6.9014416935919565]
  We jointly trained a deep network to perform image classification and to predict neural activity in macaque primary visual cortex (V1) We found that co-training on monkey V1 data leads to increased robustness despite the absence of those distortions during training. Our results also demonstrated that the network's representations become more brain-like as their robustness improves.
  arXiv  Detail & Related papers  (2021-07-29T21:55:48Z)
• Joint Learning of Neural Transfer and Architecture Adaptation for Image Recognition [77.95361323613147]
  Current state-of-the-art visual recognition systems rely on pretraining a neural network on a large-scale dataset and finetuning the network weights on a smaller dataset. In this work, we prove that dynamically adapting network architectures tailored for each domain task along with weight finetuning benefits in both efficiency and effectiveness. Our method can be easily generalized to an unsupervised paradigm by replacing supernet training with self-supervised learning in the source domain tasks and performing linear evaluation in the downstream tasks.
  arXiv  Detail & Related papers  (2021-03-31T08:15:17Z)
• Understanding the Role of Individual Units in a Deep Neural Network [85.23117441162772]
  We present an analytic framework to systematically identify hidden units within image classification and image generation networks. First, we analyze a convolutional neural network (CNN) trained on scene classification and discover units that match a diverse set of object concepts. Second, we use a similar analytic method to analyze a generative adversarial network (GAN) model trained to generate scenes.
  arXiv  Detail & Related papers  (2020-09-10T17:59:10Z)
• Neural Sparse Representation for Image Restoration [116.72107034624344]
  Inspired by the robustness and efficiency of sparse coding based image restoration models, we investigate the sparsity of neurons in deep networks. Our method structurally enforces sparsity constraints upon hidden neurons. Experiments show that sparse representation is crucial in deep neural networks for multiple image restoration tasks.
  arXiv  Detail & Related papers  (2020-06-08T05:15:17Z)
• Exploiting Semantics for Face Image Deblurring [121.44928934662063]
  We propose an effective and efficient face deblurring algorithm by exploiting semantic cues via deep convolutional neural networks. We incorporate face semantic labels as input priors and propose an adaptive structural loss to regularize facial local structures. The proposed method restores sharp images with more accurate facial features and details.
  arXiv  Detail & Related papers  (2020-01-19T13:06:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.