Shortcut Learning in Deep Neural Networks
- URL: http://arxiv.org/abs/2004.07780v5
- Date: Tue, 21 Nov 2023 15:22:43 GMT
- Title: Shortcut Learning in Deep Neural Networks
- Authors: Robert Geirhos, J\"orn-Henrik Jacobsen, Claudio Michaelis, Richard
Zemel, Wieland Brendel, Matthias Bethge, Felix A. Wichmann
- Abstract summary: We seek to distill how many of deep learning's problems can be seen as different symptoms of the same underlying problem: shortcut learning.
Shortcuts are decision rules that perform well on standard benchmarks but fail to transfer to more challenging testing conditions, such as real-world scenarios.
We develop recommendations for model interpretation and benchmarking, highlighting recent advances in machine learning to improve robustness and transferability from the lab to real-world applications.
- Score: 29.088631285225237
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Deep learning has triggered the current rise of artificial intelligence and
is the workhorse of today's machine intelligence. Numerous success stories have
rapidly spread all over science, industry and society, but its limitations have
only recently come into focus. In this perspective we seek to distill how many
of deep learning's problems can be seen as different symptoms of the same
underlying problem: shortcut learning. Shortcuts are decision rules that
perform well on standard benchmarks but fail to transfer to more challenging
testing conditions, such as real-world scenarios. Related issues are known in
Comparative Psychology, Education and Linguistics, suggesting that shortcut
learning may be a common characteristic of learning systems, biological and
artificial alike. Based on these observations, we develop a set of
recommendations for model interpretation and benchmarking, highlighting recent
advances in machine learning to improve robustness and transferability from the
lab to real-world applications.
Related papers
- A Review of Neuroscience-Inspired Machine Learning [58.72729525961739]
Bio-plausible credit assignment is compatible with practically any learning condition and is energy-efficient.
In this paper, we survey several vital algorithms that model bio-plausible rules of credit assignment in artificial neural networks.
We conclude by discussing the future challenges that will need to be addressed in order to make such algorithms more useful in practical applications.
arXiv Detail & Related papers (2024-02-16T18:05:09Z) - Brain-Inspired Computational Intelligence via Predictive Coding [89.6335791546526]
Predictive coding (PC) has shown promising performance in machine intelligence tasks.
PC can model information processing in different brain areas, can be used in cognitive control and robotics.
arXiv Detail & Related papers (2023-08-15T16:37:16Z) - Machine Unlearning: A Survey [56.79152190680552]
A special need has arisen where, due to privacy, usability, and/or the right to be forgotten, information about some specific samples needs to be removed from a model, called machine unlearning.
This emerging technology has drawn significant interest from both academics and industry due to its innovation and practicality.
No study has analyzed this complex topic or compared the feasibility of existing unlearning solutions in different kinds of scenarios.
The survey concludes by highlighting some of the outstanding issues with unlearning techniques, along with some feasible directions for new research opportunities.
arXiv Detail & Related papers (2023-06-06T10:18:36Z) - Towards Understanding How Machines Can Learn Causal Overhypotheses [4.540122114051773]
Children are adept at many kinds of causal inference and learning.
One of the key challenges for current machine learning algorithms is modeling and understanding causal overhypotheses.
We present a new benchmark -- a flexible environment which allows for the evaluation of existing techniques.
arXiv Detail & Related papers (2022-06-16T17:54:16Z) - Towards continual task learning in artificial neural networks: current
approaches and insights from neuroscience [0.0]
The innate capacity of humans and other animals to learn a diverse, and often interfering, range of knowledge is a hallmark of natural intelligence.
The ability of artificial neural networks to learn across a range of tasks and domains is a clear goal of artificial intelligence.
arXiv Detail & Related papers (2021-12-28T13:50:51Z) - From Machine Learning to Robotics: Challenges and Opportunities for
Embodied Intelligence [113.06484656032978]
Article argues that embodied intelligence is a key driver for the advancement of machine learning technology.
We highlight challenges and opportunities specific to embodied intelligence.
We propose research directions which may significantly advance the state-of-the-art in robot learning.
arXiv Detail & Related papers (2021-10-28T16:04:01Z) - Ten Quick Tips for Deep Learning in Biology [116.78436313026478]
Machine learning is concerned with the development and applications of algorithms that can recognize patterns in data and use them for predictive modeling.
Deep learning has become its own subfield of machine learning.
In the context of biological research, deep learning has been increasingly used to derive novel insights from high-dimensional biological data.
arXiv Detail & Related papers (2021-05-29T21:02:44Z) - Knowledge as Invariance -- History and Perspectives of
Knowledge-augmented Machine Learning [69.99522650448213]
Research in machine learning is at a turning point.
Research interests are shifting away from increasing the performance of highly parameterized models to exceedingly specific tasks.
This white paper provides an introduction and discussion of this emerging field in machine learning research.
arXiv Detail & Related papers (2020-12-21T15:07:19Z) - Off-the-shelf deep learning is not enough: parsimony, Bayes and
causality [0.8602553195689513]
We discuss opportunities and roadblocks to implementation of deep learning within materials science.
We argue that deep learning and AI are now well positioned to revolutionize fields where causal links are known.
arXiv Detail & Related papers (2020-05-04T15:16:30Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.