A Brain-Inspired Sequence Learning Model based on a Logic

• URL: http://arxiv.org/abs/2308.12486v2
• Date: Mon, 6 Nov 2023 16:26:09 GMT
• Title: A Brain-Inspired Sequence Learning Model based on a Logic
• Authors: Bowen Xu
• Abstract summary: In this paper, a model of sequence learning, which is interpretable through Non-Axiomatic Logic, is designed and tested. The results show that the model works well within different levels of difficulty.
• Score: 6.653734987585243
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Sequence learning is an essential aspect of intelligence. In Artificial Intelligence, sequence prediction task is usually used to test a sequence learning model. In this paper, a model of sequence learning, which is interpretable through Non-Axiomatic Logic, is designed and tested. The learning mechanism is composed of three steps, hypothesizing, revising, and recycling, which enable the model to work under the Assumption of Insufficient Knowledge and Resources. Synthetic datasets for sequence prediction task are generated to test the capacity of the model. The results show that the model works well within different levels of difficulty. In addition, since the model adopts concept-centered representation, it theoretically does not suffer from catastrophic forgetting, and the practical results also support this property. This paper shows the potential of learning sequences in a logical way.

Related papers

• Deep Learning Through A Telescoping Lens: A Simple Model Provides Empirical Insights On Grokking, Gradient Boosting & Beyond [61.18736646013446]
  In pursuit of a deeper understanding of its surprising behaviors, we investigate the utility of a simple yet accurate model of a trained neural network. Across three case studies, we illustrate how it can be applied to derive new empirical insights on a diverse range of prominent phenomena.
  arXiv  Detail & Related papers  (2024-10-31T22:54:34Z)
• Action Model Learning with Guarantees [5.524804393257921]
  We develop a theory for action model learning based on version spaces that interprets the task as search for hypothesis that are consistent with the learning examples. Our theoretical findings are instantiated in an online algorithm that maintains a compact representation of all solutions of the problem.
  arXiv  Detail & Related papers  (2024-04-15T10:01:43Z)
• Generative Models as a Complex Systems Science: How can we make sense of large language model behavior? [75.79305790453654]
  Coaxing out desired behavior from pretrained models, while avoiding undesirable ones, has redefined NLP. We argue for a systematic effort to decompose language model behavior into categories that explain cross-task performance.
  arXiv  Detail & Related papers  (2023-07-31T22:58:41Z)
• A Recursive Bateson-Inspired Model for the Generation of Semantic Formal Concepts from Spatial Sensory Data [77.34726150561087]
  This paper presents a new symbolic-only method for the generation of hierarchical concept structures from complex sensory data. The approach is based on Bateson's notion of difference as the key to the genesis of an idea or a concept. The model is able to produce fairly rich yet human-readable conceptual representations without training.
  arXiv  Detail & Related papers  (2023-07-16T15:59:13Z)
• Computation with Sequences in a Model of the Brain [11.15191997898358]
  How cognition arises from neural activity is a central open question in neuroscience. We show that time can be captured naturally as precedence through synaptic weights and plasticity. We show that any finite state machine can be learned in a similar way, through the presentation of appropriate patterns of sequences.
  arXiv  Detail & Related papers  (2023-06-06T15:58:09Z)
• Robust Graph Representation Learning via Predictive Coding [46.22695915912123]
  Predictive coding is a message-passing framework initially developed to model information processing in the brain. In this work, we build models that rely on the message-passing rule of predictive coding. We show that the proposed models are comparable to standard ones in terms of performance in both inductive and transductive tasks.
  arXiv  Detail & Related papers  (2022-12-09T03:58:22Z)
• Learning to Reason With Relational Abstractions [65.89553417442049]
  We study how to build stronger reasoning capability in language models using the idea of relational abstractions. We find that models that are supplied with such sequences as prompts can solve tasks with a significantly higher accuracy.
  arXiv  Detail & Related papers  (2022-10-06T00:27:50Z)
• Learning continuous models for continuous physics [94.42705784823997]
  We develop a test based on numerical analysis theory to validate machine learning models for science and engineering applications. Our results illustrate how principled numerical analysis methods can be coupled with existing ML training/testing methodologies to validate models for science and engineering applications.
  arXiv  Detail & Related papers  (2022-02-17T07:56:46Z)
• The Causal Neural Connection: Expressiveness, Learnability, and Inference [125.57815987218756]
  An object called structural causal model (SCM) represents a collection of mechanisms and sources of random variation of the system under investigation. In this paper, we show that the causal hierarchy theorem (Thm. 1, Bareinboim et al., 2020) still holds for neural models. We introduce a special type of SCM called a neural causal model (NCM), and formalize a new type of inductive bias to encode structural constraints necessary for performing causal inferences.
  arXiv  Detail & Related papers  (2021-07-02T01:55:18Z)
• Demystifying Code Summarization Models [5.608277537412537]
  We evaluate four prominent code summarization models: extreme summarizer, code2vec, code2seq, and sequence GNN. Results show that all models base their predictions on syntactic and lexical properties with little to none semantic implication. We present a novel approach to explaining the predictions of code summarization models through the lens of training data.
  arXiv  Detail & Related papers  (2021-02-09T03:17:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.