A Memory-Augmented Neural Network Model of Abstract Rule Learning
- URL: http://arxiv.org/abs/2012.07172v2
- Date: Tue, 15 Dec 2020 03:35:19 GMT
- Title: A Memory-Augmented Neural Network Model of Abstract Rule Learning
- Authors: Ishan Sinha, Taylor W. Webb, Jonathan D. Cohen
- Abstract summary: We focus on neural networks' capacity for arbitrary role-filler binding.
We introduce the Emergent Symbol Binding Network (ESBN), a recurrent neural network model that learns to use an external memory as a binding mechanism.
This mechanism enables symbol-like variable representations to emerge through the ESBN's training process without the need for explicit symbol-processing machinery.
- Score: 2.3562267625320352
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Human intelligence is characterized by a remarkable ability to infer abstract
rules from experience and apply these rules to novel domains. As such,
designing neural network algorithms with this capacity is an important step
toward the development of deep learning systems with more human-like
intelligence. However, doing so is a major outstanding challenge, one that some
argue will require neural networks to use explicit symbol-processing
mechanisms. In this work, we focus on neural networks' capacity for arbitrary
role-filler binding, the ability to associate abstract "roles" to
context-specific "fillers," which many have argued is an important mechanism
underlying the ability to learn and apply rules abstractly. Using a simplified
version of Raven's Progressive Matrices, a hallmark test of human intelligence,
we introduce a sequential formulation of a visual problem-solving task that
requires this form of binding. Further, we introduce the Emergent Symbol
Binding Network (ESBN), a recurrent neural network model that learns to use an
external memory as a binding mechanism. This mechanism enables symbol-like
variable representations to emerge through the ESBN's training process without
the need for explicit symbol-processing machinery. We empirically demonstrate
that the ESBN successfully learns the underlying abstract rule structure of our
task and perfectly generalizes this rule structure to novel fillers.
Related papers
- VisualPredicator: Learning Abstract World Models with Neuro-Symbolic Predicates for Robot Planning [86.59849798539312]
We present Neuro-Symbolic Predicates, a first-order abstraction language that combines the strengths of symbolic and neural knowledge representations.
We show that our approach offers better sample complexity, stronger out-of-distribution generalization, and improved interpretability.
arXiv Detail & Related papers (2024-10-30T16:11:05Z) - Coding schemes in neural networks learning classification tasks [52.22978725954347]
We investigate fully-connected, wide neural networks learning classification tasks.
We show that the networks acquire strong, data-dependent features.
Surprisingly, the nature of the internal representations depends crucially on the neuronal nonlinearity.
arXiv Detail & Related papers (2024-06-24T14:50:05Z) - Towards Scalable and Versatile Weight Space Learning [51.78426981947659]
This paper introduces the SANE approach to weight-space learning.
Our method extends the idea of hyper-representations towards sequential processing of subsets of neural network weights.
arXiv Detail & Related papers (2024-06-14T13:12:07Z) - The Role of Foundation Models in Neuro-Symbolic Learning and Reasoning [54.56905063752427]
Neuro-Symbolic AI (NeSy) holds promise to ensure the safe deployment of AI systems.
Existing pipelines that train the neural and symbolic components sequentially require extensive labelling.
New architecture, NeSyGPT, fine-tunes a vision-language foundation model to extract symbolic features from raw data.
arXiv Detail & Related papers (2024-02-02T20:33:14Z) - Emergence of Symbols in Neural Networks for Semantic Understanding and
Communication [8.156761369660096]
We propose a solution to endow neural networks with the ability to create symbols, understand semantics, and achieve communication.
SEA-net generates symbols that dynamically configure the network to perform specific tasks.
These symbols capture compositional semantic information that allows the system to acquire new functions purely by symbolic manipulation or communication.
arXiv Detail & Related papers (2023-04-13T10:13:00Z) - Knowledge-based Analogical Reasoning in Neuro-symbolic Latent Spaces [20.260546238369205]
We propose a framework that combines the pattern recognition abilities of neural networks with symbolic reasoning and background knowledge.
We take inspiration from the 'neural algorithmic reasoning' approach [DeepMind 2020] and use problem-specific background knowledge.
We test this on visual analogy problems in RAVENs Progressive Matrices, and achieve accuracy competitive with human performance.
arXiv Detail & Related papers (2022-09-19T04:03:20Z) - Emergence of Machine Language: Towards Symbolic Intelligence with Neural
Networks [73.94290462239061]
We propose to combine symbolism and connectionism principles by using neural networks to derive a discrete representation.
By designing an interactive environment and task, we demonstrated that machines could generate a spontaneous, flexible, and semantic language.
arXiv Detail & Related papers (2022-01-14T14:54:58Z) - Learning Continuous Chaotic Attractors with a Reservoir Computer [0.0]
We train a 1000-neuron RNN to abstract a continuous dynamical attractor memory from isolated examples of dynamical attractor memories.
By training the RC on isolated and shifted examples of either stable limit cycles or chaotic Lorenz attractors, the RC learns a continuum of attractors, as quantified by an extra Lyapunov exponent equal to zero.
arXiv Detail & Related papers (2021-10-16T18:07:27Z) - A neural anisotropic view of underspecification in deep learning [60.119023683371736]
We show that the way neural networks handle the underspecification of problems is highly dependent on the data representation.
Our results highlight that understanding the architectural inductive bias in deep learning is fundamental to address the fairness, robustness, and generalization of these systems.
arXiv Detail & Related papers (2021-04-29T14:31:09Z) - Emergent Symbols through Binding in External Memory [2.3562267625320352]
We introduce the Emergent Symbol Binding Network (ESBN), a recurrent network augmented with an external memory.
This binding mechanism allows symbol-like representations to emerge through the learning process without the need to explicitly incorporate symbol-processing machinery.
Across a series of tasks, we show that this architecture displays nearly perfect generalization of learned rules to novel entities.
arXiv Detail & Related papers (2020-12-29T04:28:32Z) - A neural network model of perception and reasoning [0.0]
We show that a simple set of biologically consistent organizing principles confer these capabilities to neuronal networks.
We implement these principles in a novel machine learning algorithm, based on concept construction instead of optimization, to design deep neural networks that reason with explainable neuron activity.
arXiv Detail & Related papers (2020-02-26T06:26:04Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.