Human Eyes Inspired Recurrent Neural Networks are More Robust Against
Adversarial Noises
- URL: http://arxiv.org/abs/2206.07282v1
- Date: Wed, 15 Jun 2022 03:44:42 GMT
- Title: Human Eyes Inspired Recurrent Neural Networks are More Robust Against
Adversarial Noises
- Authors: Minkyu Choi, Yizhen Zhang, Kuan Han, Xiaokai Wang, and Zhongming Liu
- Abstract summary: Compared to human vision, computer vision based on convolutional neural networks (CNN) are more vulnerable to adversarial noises.
This difference is likely attributable to how the eyes sample visual input and how the brain processes retinal samples through its dorsal and ventral visual pathways.
We design recurrent neural networks, including an input sampler that mimics the human retina, a dorsal network that guides where to look next, and a ventral network that represents the retinal samples.
Taking these modules together, the models learn to take multiple glances at an image, attend to a salient part at each glance, and accumulate the representation over time to recognize the image.
- Score: 3.8738982761490988
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Compared to human vision, computer vision based on convolutional neural
networks (CNN) are more vulnerable to adversarial noises. This difference is
likely attributable to how the eyes sample visual input and how the brain
processes retinal samples through its dorsal and ventral visual pathways, which
are under-explored for computer vision. Inspired by the brain, we design
recurrent neural networks, including an input sampler that mimics the human
retina, a dorsal network that guides where to look next, and a ventral network
that represents the retinal samples. Taking these modules together, the models
learn to take multiple glances at an image, attend to a salient part at each
glance, and accumulate the representation over time to recognize the image. We
test such models for their robustness against a varying level of adversarial
noises with a special focus on the effect of different input sampling
strategies. Our findings suggest that retinal foveation and sampling renders a
model more robust against adversarial noises, and the model may correct itself
from an attack when it is given a longer time to take more glances at an image.
In conclusion, robust visual recognition can benefit from the combined use of
three brain-inspired mechanisms: retinal transformation, attention guided eye
movement, and recurrent processing, as opposed to feedforward-only CNNs.
Related papers
- A Dual-Stream Neural Network Explains the Functional Segregation of
Dorsal and Ventral Visual Pathways in Human Brains [8.24969449883056]
We develop a dual-stream vision model inspired by the human eyes and brain.
At the input level, the model samples two complementary visual patterns.
At the backend, the model processes the separate input patterns through two branches of convolutional neural networks.
arXiv Detail & Related papers (2023-10-20T22:47:40Z) - Training on Foveated Images Improves Robustness to Adversarial Attacks [26.472800216546233]
Deep neural networks (DNNs) have been shown to be vulnerable to adversarial attacks.
RBlur is an image transform that simulates the loss in fidelity of peripheral vision by blurring the image and reducing its color saturation.
DNNs trained on images transformed by RBlur are substantially more robust to adversarial attacks, as well as other, non-adversarial, corruptions, achieving up to 25% higher accuracy on perturbed data.
arXiv Detail & Related papers (2023-08-01T21:40:30Z) - Probing neural representations of scene perception in a hippocampally
dependent task using artificial neural networks [1.0312968200748116]
Deep artificial neural networks (DNNs) trained through backpropagation provide effective models of the mammalian visual system.
We describe a novel scene perception benchmark inspired by a hippocampal dependent task.
Using a network architecture inspired by the connectivity between temporal lobe structures and the hippocampus, we demonstrate that DNNs trained using a triplet loss can learn this task.
arXiv Detail & Related papers (2023-03-11T10:26:25Z) - BI AVAN: Brain inspired Adversarial Visual Attention Network [67.05560966998559]
We propose a brain-inspired adversarial visual attention network (BI-AVAN) to characterize human visual attention directly from functional brain activity.
Our model imitates the biased competition process between attention-related/neglected objects to identify and locate the visual objects in a movie frame the human brain focuses on in an unsupervised manner.
arXiv Detail & Related papers (2022-10-27T22:20:36Z) - Adapting Brain-Like Neural Networks for Modeling Cortical Visual
Prostheses [68.96380145211093]
Cortical prostheses are devices implanted in the visual cortex that attempt to restore lost vision by electrically stimulating neurons.
Currently, the vision provided by these devices is limited, and accurately predicting the visual percepts resulting from stimulation is an open challenge.
We propose to address this challenge by utilizing 'brain-like' convolutional neural networks (CNNs), which have emerged as promising models of the visual system.
arXiv Detail & Related papers (2022-09-27T17:33:19Z) - Peripheral Vision Transformer [52.55309200601883]
We take a biologically inspired approach and explore to model peripheral vision in deep neural networks for visual recognition.
We propose to incorporate peripheral position encoding to the multi-head self-attention layers to let the network learn to partition the visual field into diverse peripheral regions given training data.
We evaluate the proposed network, dubbed PerViT, on the large-scale ImageNet dataset and systematically investigate the inner workings of the model for machine perception.
arXiv Detail & Related papers (2022-06-14T12:47:47Z) - Searching for the Essence of Adversarial Perturbations [73.96215665913797]
We show that adversarial perturbations contain human-recognizable information, which is the key conspirator responsible for a neural network's erroneous prediction.
This concept of human-recognizable information allows us to explain key features related to adversarial perturbations.
arXiv Detail & Related papers (2022-05-30T18:04:57Z) - Prune and distill: similar reformatting of image information along rat
visual cortex and deep neural networks [61.60177890353585]
Deep convolutional neural networks (CNNs) have been shown to provide excellent models for its functional analogue in the brain, the ventral stream in visual cortex.
Here we consider some prominent statistical patterns that are known to exist in the internal representations of either CNNs or the visual cortex.
We show that CNNs and visual cortex share a similarly tight relationship between dimensionality expansion/reduction of object representations and reformatting of image information.
arXiv Detail & Related papers (2022-05-27T08:06:40Z) - Towards robust vision by multi-task learning on monkey visual cortex [6.9014416935919565]
We jointly trained a deep network to perform image classification and to predict neural activity in macaque primary visual cortex (V1)
We found that co-training on monkey V1 data leads to increased robustness despite the absence of those distortions during training.
Our results also demonstrated that the network's representations become more brain-like as their robustness improves.
arXiv Detail & Related papers (2021-07-29T21:55:48Z) - Fooling the primate brain with minimal, targeted image manipulation [67.78919304747498]
We propose an array of methods for creating minimal, targeted image perturbations that lead to changes in both neuronal activity and perception as reflected in behavior.
Our work shares the same goal with adversarial attack, namely the manipulation of images with minimal, targeted noise that leads ANN models to misclassify the images.
arXiv Detail & Related papers (2020-11-11T08:30:54Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.