Learning a Deep Generative Model like a Program: the Free Category Prior
- URL: http://arxiv.org/abs/2011.11063v1
- Date: Sun, 22 Nov 2020 17:16:17 GMT
- Title: Learning a Deep Generative Model like a Program: the Free Category Prior
- Authors: Eli Sennesh
- Abstract summary: We show how our formalism allows neural networks to serve as primitives in probabilistic programs.
We show how our formalism allows neural networks to serve as primitives in probabilistic programs.
- Score: 2.088583843514496
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Humans surpass the cognitive abilities of most other animals in our ability
to "chunk" concepts into words, and then combine the words to combine the
concepts. In this process, we make "infinite use of finite means", enabling us
to learn new concepts quickly and nest concepts within each-other. While
program induction and synthesis remain at the heart of foundational theories of
artificial intelligence, only recently has the community moved forward in
attempting to use program learning as a benchmark task itself. The cognitive
science community has thus often assumed that if the brain has simulation and
reasoning capabilities equivalent to a universal computer, then it must employ
a serialized, symbolic representation. Here we confront that assumption, and
provide a counterexample in which compositionality is expressed via network
structure: the free category prior over programs. We show how our formalism
allows neural networks to serve as primitives in probabilistic programs. We
learn both program structure and model parameters end-to-end.
Related papers
- OC-NMN: Object-centric Compositional Neural Module Network for
Generative Visual Analogical Reasoning [49.12350554270196]
We show how modularity can be leveraged to derive a compositional data augmentation framework inspired by imagination.
Our method, denoted Object-centric Compositional Neural Module Network (OC-NMN), decomposes visual generative reasoning tasks into a series of primitives applied to objects without using a domain-specific language.
arXiv Detail & Related papers (2023-10-28T20:12:58Z) - A Recursive Bateson-Inspired Model for the Generation of Semantic Formal
Concepts from Spatial Sensory Data [77.34726150561087]
This paper presents a new symbolic-only method for the generation of hierarchical concept structures from complex sensory data.
The approach is based on Bateson's notion of difference as the key to the genesis of an idea or a concept.
The model is able to produce fairly rich yet human-readable conceptual representations without training.
arXiv Detail & Related papers (2023-07-16T15:59:13Z) - From Word Models to World Models: Translating from Natural Language to
the Probabilistic Language of Thought [124.40905824051079]
We propose rational meaning construction, a computational framework for language-informed thinking.
We frame linguistic meaning as a context-sensitive mapping from natural language into a probabilistic language of thought.
We show that LLMs can generate context-sensitive translations that capture pragmatically-appropriate linguistic meanings.
We extend our framework to integrate cognitively-motivated symbolic modules.
arXiv Detail & Related papers (2023-06-22T05:14:00Z) - A Neural Lambda Calculus: Neurosymbolic AI meets the foundations of
computing and functional programming [0.0]
We will analyze the ability of neural networks to learn how to execute programs as a whole.
We will introduce the use of integrated neural learning and calculi formalization.
arXiv Detail & Related papers (2023-04-18T20:30:16Z) - An Enactivist-Inspired Mathematical Model of Cognition [5.8010446129208155]
We formulate five basic tenets of enactivist cognitive science that we have carefully identified in the relevant literature.
We then develop a mathematical framework to talk about cognitive systems which complies with these enactivist tenets.
arXiv Detail & Related papers (2022-06-10T13:03:47Z) - Emergence of Machine Language: Towards Symbolic Intelligence with Neural
Networks [73.94290462239061]
We propose to combine symbolism and connectionism principles by using neural networks to derive a discrete representation.
By designing an interactive environment and task, we demonstrated that machines could generate a spontaneous, flexible, and semantic language.
arXiv Detail & Related papers (2022-01-14T14:54:58Z) - A Mathematical Approach to Constraining Neural Abstraction and the
Mechanisms Needed to Scale to Higher-Order Cognition [0.0]
Artificial intelligence has made great strides in the last decade but still falls short of the human brain, the best-known example of intelligence.
Not much is known of the neural processes that allow the brain to make the leap to achieve so much from so little.
This paper proposes a mathematical approach using graph theory and spectral graph theory, to hypothesize how to constrain these neural clusters of information.
arXiv Detail & Related papers (2021-08-12T02:13:22Z) - Compositional Processing Emerges in Neural Networks Solving Math
Problems [100.80518350845668]
Recent progress in artificial neural networks has shown that when large models are trained on enough linguistic data, grammatical structure emerges in their representations.
We extend this work to the domain of mathematical reasoning, where it is possible to formulate precise hypotheses about how meanings should be composed.
Our work shows that neural networks are not only able to infer something about the structured relationships implicit in their training data, but can also deploy this knowledge to guide the composition of individual meanings into composite wholes.
arXiv Detail & Related papers (2021-05-19T07:24:42Z) - Inductive Biases for Deep Learning of Higher-Level Cognition [108.89281493851358]
A fascinating hypothesis is that human and animal intelligence could be explained by a few principles.
This work considers a larger list, focusing on those which concern mostly higher-level and sequential conscious processing.
The objective of clarifying these particular principles is that they could potentially help us build AI systems benefiting from humans' abilities.
arXiv Detail & Related papers (2020-11-30T18:29:25Z) - Learning Hierarchically Structured Concepts [3.9795499448909024]
We show how a biologically plausible neural network can recognize hierarchically structured concepts.
For learning, we analyze Oja's rule formally, a well-known biologically-plausible rule for adjusting the weights of synapses.
We complement the learning results with lower bounds asserting that, in order to recognize concepts of a certain hierarchical depth, neural networks must have a corresponding number of layers.
arXiv Detail & Related papers (2019-09-10T15:11:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.