Inductive Biases for Deep Learning of Higher-Level Cognition
- URL: http://arxiv.org/abs/2011.15091v3
- Date: Wed, 17 Feb 2021 21:54:35 GMT
- Title: Inductive Biases for Deep Learning of Higher-Level Cognition
- Authors: Anirudh Goyal, Yoshua Bengio
- Abstract summary: A fascinating hypothesis is that human and animal intelligence could be explained by a few principles.
This work considers a larger list, focusing on those which concern mostly higher-level and sequential conscious processing.
The objective of clarifying these particular principles is that they could potentially help us build AI systems benefiting from humans' abilities.
- Score: 108.89281493851358
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: A fascinating hypothesis is that human and animal intelligence could be
explained by a few principles (rather than an encyclopedic list of heuristics).
If that hypothesis was correct, we could more easily both understand our own
intelligence and build intelligent machines. Just like in physics, the
principles themselves would not be sufficient to predict the behavior of
complex systems like brains, and substantial computation might be needed to
simulate human-like intelligence. This hypothesis would suggest that studying
the kind of inductive biases that humans and animals exploit could help both
clarify these principles and provide inspiration for AI research and
neuroscience theories. Deep learning already exploits several key inductive
biases, and this work considers a larger list, focusing on those which concern
mostly higher-level and sequential conscious processing. The objective of
clarifying these particular principles is that they could potentially help us
build AI systems benefiting from humans' abilities in terms of flexible
out-of-distribution and systematic generalization, which is currently an area
where a large gap exists between state-of-the-art machine learning and human
intelligence.
Related papers
- Machine learning and information theory concepts towards an AI
Mathematician [77.63761356203105]
The current state-of-the-art in artificial intelligence is impressive, especially in terms of mastery of language, but not so much in terms of mathematical reasoning.
This essay builds on the idea that current deep learning mostly succeeds at system 1 abilities.
It takes an information-theoretical posture to ask questions about what constitutes an interesting mathematical statement.
arXiv Detail & Related papers (2024-03-07T15:12:06Z) - The Generative AI Paradox: "What It Can Create, It May Not Understand" [81.89252713236746]
Recent wave of generative AI has sparked excitement and concern over potentially superhuman levels of artificial intelligence.
At the same time, models still show basic errors in understanding that would not be expected even in non-expert humans.
This presents us with an apparent paradox: how do we reconcile seemingly superhuman capabilities with the persistence of errors that few humans would make?
arXiv Detail & Related papers (2023-10-31T18:07:07Z) - AI for Mathematics: A Cognitive Science Perspective [86.02346372284292]
Mathematics is one of the most powerful conceptual systems developed and used by the human species.
Rapid progress in AI, particularly propelled by advances in large language models (LLMs), has sparked renewed, widespread interest in building such systems.
arXiv Detail & Related papers (2023-10-19T02:00:31Z) - Brain-Inspired Computational Intelligence via Predictive Coding [89.6335791546526]
Predictive coding (PC) has shown promising performance in machine intelligence tasks.
PC can model information processing in different brain areas, can be used in cognitive control and robotics.
arXiv Detail & Related papers (2023-08-15T16:37:16Z) - The Nature of Intelligence [0.0]
The essence of intelligence commonly represented by both humans and AI is unknown.
We show that the nature of intelligence is a series of mathematically functional processes that minimize system entropy.
This essay should be a starting point for a deeper understanding of the universe and us as human beings.
arXiv Detail & Related papers (2023-07-20T23:11:59Z) - A Mathematical Approach to Constraining Neural Abstraction and the
Mechanisms Needed to Scale to Higher-Order Cognition [0.0]
Artificial intelligence has made great strides in the last decade but still falls short of the human brain, the best-known example of intelligence.
Not much is known of the neural processes that allow the brain to make the leap to achieve so much from so little.
This paper proposes a mathematical approach using graph theory and spectral graph theory, to hypothesize how to constrain these neural clusters of information.
arXiv Detail & Related papers (2021-08-12T02:13:22Z) - Applying Deutsch's concept of good explanations to artificial
intelligence and neuroscience -- an initial exploration [0.0]
We investigate Deutsch's hard-to-vary principle and how it relates to more formalized principles in deep learning.
We look at what role hard-tovary explanations play in intelligence by looking at the human brain.
arXiv Detail & Related papers (2020-12-16T23:23:22Z) - Machine Common Sense [77.34726150561087]
Machine common sense remains a broad, potentially unbounded problem in artificial intelligence (AI)
This article deals with the aspects of modeling commonsense reasoning focusing on such domain as interpersonal interactions.
arXiv Detail & Related papers (2020-06-15T13:59:47Z) - Recursion, evolution and conscious self [0.0]
We study a learning theory which is roughly automatic, that is, it does not require but a minimum of initial programming.
The conclusions agree with scientific findings in both biology and neuroscience.
arXiv Detail & Related papers (2020-01-14T11:04:52Z) - Is Intelligence Artificial? [0.0]
This paper attempts to give a unifying definition that can be applied to the natural world in general and then Artificial Intelligence.
A metric that is grounded in Kolmogorov's Complexity Theory is suggested, which leads to a measurement about entropy.
A version of an accepted AI test is then put forward as the 'acid test' and might be what a free-thinking program would try to achieve.
arXiv Detail & Related papers (2014-03-05T11:09:55Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.