Abutting Grating Illusion: Cognitive Challenge to Neural Network Models
- URL: http://arxiv.org/abs/2208.03958v1
- Date: Mon, 8 Aug 2022 08:01:11 GMT
- Title: Abutting Grating Illusion: Cognitive Challenge to Neural Network Models
- Authors: Jinyu Fan and Yi Zeng
- Abstract summary: We propose a novel corruption method based on the abutting grating illusion.
The method destroys the gradient-defined boundaries and generates the perception of illusory contours using line gratings abutting each other.
Various deep learning models are tested on the corruption, including models trained from scratch and 109 models pretrained with ImageNet or various data augmentation techniques.
- Score: 4.031522806737616
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Even the state-of-the-art deep learning models lack fundamental abilities
compared to humans. Multiple comparison paradigms have been proposed to explore
the distinctions between humans and deep learning. While most comparisons rely
on corruptions inspired by mathematical transformations, very few have bases on
human cognitive phenomena. In this study, we propose a novel corruption method
based on the abutting grating illusion, which is a visual phenomenon widely
discovered in both human and a wide range of animal species. The corruption
method destroys the gradient-defined boundaries and generates the perception of
illusory contours using line gratings abutting each other. We applied the
method on MNIST, high resolution MNIST, and silhouette object images. Various
deep learning models are tested on the corruption, including models trained
from scratch and 109 models pretrained with ImageNet or various data
augmentation techniques. Our results show that abutting grating corruption is
challenging even for state-of-the-art deep learning models because most models
are randomly guessing. We also discovered that the DeepAugment technique can
greatly improve robustness against abutting grating illusion. Visualisation of
early layers indicates that better performing models exhibit stronger
end-stopping property, which is consistent with neuroscience discoveries. To
validate the corruption method, 24 human subjects are involved to classify
samples of corrupted datasets.
Related papers
- Neural feels with neural fields: Visuo-tactile perception for in-hand
manipulation [57.60490773016364]
We combine vision and touch sensing on a multi-fingered hand to estimate an object's pose and shape during in-hand manipulation.
Our method, NeuralFeels, encodes object geometry by learning a neural field online and jointly tracks it by optimizing a pose graph problem.
Our results demonstrate that touch, at the very least, refines and, at the very best, disambiguates visual estimates during in-hand manipulation.
arXiv Detail & Related papers (2023-12-20T22:36:37Z) - Detecting Generated Images by Real Images Only [64.12501227493765]
Existing generated image detection methods detect visual artifacts in generated images or learn discriminative features from both real and generated images by massive training.
This paper approaches the generated image detection problem from a new perspective: Start from real images.
By finding the commonality of real images and mapping them to a dense subspace in feature space, the goal is that generated images, regardless of their generative model, are then projected outside the subspace.
arXiv Detail & Related papers (2023-11-02T03:09:37Z) - Intriguing properties of generative classifiers [14.57861413242093]
We build on advances in generative modeling that turn text-to-image models into classifiers.
They show a record-breaking human-like shape bias (99% for Imagen), near human-level out-of-distribution accuracy, state-of-the-art alignment with human classification errors.
Our results indicate that while the current dominant paradigm for modeling human object recognition is discriminative inference, zero-shot generative models approximate human object recognition data surprisingly well.
arXiv Detail & Related papers (2023-09-28T18:19:40Z) - Hierarchical Contrastive Learning for Pattern-Generalizable Image
Corruption Detection [40.04083743934034]
We develop a hierarchical contrastive learning framework to detect corrupted regions.
A specialized hierarchical interaction mechanism is designed to facilitate the knowledge of contrastive learning in different scales.
Our model has well generalization ability across different corruption patterns.
arXiv Detail & Related papers (2023-08-27T10:03:48Z) - Frequency-Based Vulnerability Analysis of Deep Learning Models against
Image Corruptions [48.34142457385199]
We present MUFIA, an algorithm designed to identify the specific types of corruptions that can cause models to fail.
We find that even state-of-the-art models trained to be robust against known common corruptions struggle against the low visibility-based corruptions crafted by MUFIA.
arXiv Detail & Related papers (2023-06-12T15:19:13Z) - Degraded Polygons Raise Fundamental Questions of Neural Network Perception [5.423100066629618]
We revisit the task of recovering images under degradation, first introduced over 30 years ago in the Recognition-by-Components theory of human vision.
We implement the Automated Shape Recoverability Test for rapidly generating large-scale datasets of perimeter-degraded regular polygons.
We find that neural networks' behavior on this simple task conflicts with human behavior.
arXiv Detail & Related papers (2023-06-08T06:02:39Z) - Deepfake Forensics via An Adversarial Game [99.84099103679816]
We advocate adversarial training for improving the generalization ability to both unseen facial forgeries and unseen image/video qualities.
Considering that AI-based face manipulation often leads to high-frequency artifacts that can be easily spotted by models yet difficult to generalize, we propose a new adversarial training method that attempts to blur out these specific artifacts.
arXiv Detail & Related papers (2021-03-25T02:20:08Z) - Predictive coding feedback results in perceived illusory contours in a
recurrent neural network [0.0]
We equip a deep feedforward convolutional network with brain-inspired recurrent dynamics.
We show that the perception of illusory contours could involve feedback connections.
arXiv Detail & Related papers (2021-02-03T09:07:09Z) - What Do Deep Nets Learn? Class-wise Patterns Revealed in the Input Space [88.37185513453758]
We propose a method to visualize and understand the class-wise knowledge learned by deep neural networks (DNNs) under different settings.
Our method searches for a single predictive pattern in the pixel space to represent the knowledge learned by the model for each class.
In the adversarial setting, we show that adversarially trained models tend to learn more simplified shape patterns.
arXiv Detail & Related papers (2021-01-18T06:38:41Z) - Informative Dropout for Robust Representation Learning: A Shape-bias
Perspective [84.30946377024297]
We propose a light-weight model-agnostic method, namely Informative Dropout (InfoDrop), to improve interpretability and reduce texture bias.
Specifically, we discriminate texture from shape based on local self-information in an image, and adopt a Dropout-like algorithm to decorrelate the model output from the local texture.
arXiv Detail & Related papers (2020-08-10T16:52:24Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.