Increasing the Inference and Learning Speed of Tsetlin Machines with Clause Indexing

• URL: http://arxiv.org/abs/2004.03188v1
• Date: Tue, 7 Apr 2020 08:16:07 GMT
• Title: Increasing the Inference and Learning Speed of Tsetlin Machines with Clause Indexing
• Authors: Saeed Rahimi Gorji, Ole-Christoffer Granmo, Sondre Glimsdal, Jonathan Edwards, Morten Goodwin
• Abstract summary: The Tsetlin Machine (TM) is a machine learning algorithm founded on the classical Tsetlin Automaton (TA) and game theory. We report up to 15 times faster classification and three times faster learning on MNIST and Fashion-MNIST image classification, and IMDb sentiment analysis.
• Score: 9.440900386313215
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The Tsetlin Machine (TM) is a machine learning algorithm founded on the classical Tsetlin Automaton (TA) and game theory. It further leverages frequent pattern mining and resource allocation principles to extract common patterns in the data, rather than relying on minimizing output error, which is prone to overfitting. Unlike the intertwined nature of pattern representation in neural networks, a TM decomposes problems into self-contained patterns, represented as conjunctive clauses. The clause outputs, in turn, are combined into a classification decision through summation and thresholding, akin to a logistic regression function, however, with binary weights and a unit step output function. In this paper, we exploit this hierarchical structure by introducing a novel algorithm that avoids evaluating the clauses exhaustively. Instead we use a simple look-up table that indexes the clauses on the features that falsify them. In this manner, we can quickly evaluate a large number of clauses through falsification, simply by iterating through the features and using the look-up table to eliminate those clauses that are falsified. The look-up table is further structured so that it facilitates constant time updating, thus supporting use also during learning. We report up to 15 times faster classification and three times faster learning on MNIST and Fashion-MNIST image classification, and IMDb sentiment analysis.

Related papers

• Exploring State Space and Reasoning by Elimination in Tsetlin Machines [14.150011713654331]
  The Tsetlin Machine (TM) has gained significant attention in Machine Learning (ML) TM is utilised to construct word embedding and describe target words using clauses. To enhance the descriptive capacity of these clauses, we study the concept of Reasoning by Elimination (RbE) in clauses' formulation.
  arXiv  Detail & Related papers  (2024-07-12T10:58:01Z)
• Neuro-Symbolic Temporal Point Processes [13.72758658973969]
  We introduce a neural-symbolic rule induction framework within the temporal point process model. The negative log-likelihood is the loss that guides the learning, where the explanatory logic rules and their weights are learned end-to-end. Our approach showcases notable efficiency and accuracy across synthetic and real datasets.
  arXiv  Detail & Related papers  (2024-06-06T09:52:56Z)
• Binning as a Pretext Task: Improving Self-Supervised Learning in Tabular Domains [0.565395466029518]
  We propose a novel pretext task based on the classical binning method. The idea is straightforward: reconstructing the bin indices (either orders or classes) rather than the original values. Our empirical investigations ascertain several advantages of binning.
  arXiv  Detail & Related papers  (2024-05-13T01:23:14Z)
• Boosting gets full Attention for Relational Learning [27.82663283409287]
  We introduce an attention mechanism for structured data that blends well with tree-based models in the training context of (gradient) boosting. Experiments on simulated and real-world domains display the competitiveness of our method against a state of the art containing both tree-based and neural nets-based models.
  arXiv  Detail & Related papers  (2024-02-22T19:16:01Z)
• Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understanding [79.9461269253121]
  We propose the Chain-of-Table framework, where tabular data is explicitly used in the reasoning chain as a proxy for intermediate thoughts. Chain-of-Table achieves new state-of-the-art performance on WikiTQ, FeTaQA, and TabFact benchmarks.
  arXiv  Detail & Related papers  (2024-01-09T07:46:26Z)
• Mitigating the Learning Bias towards Repetition by Self-Contrastive Training for Open-Ended Generation [92.42032403795879]
  We show that pretrained language models (LMs) such as GPT2 still tend to generate repetitive texts. We attribute their overestimation of token-level repetition probabilities to the learning bias. We find that LMs use longer-range dependencies to predict repetitive tokens than non-repetitive ones, which may be the cause of sentence-level repetition loops.
  arXiv  Detail & Related papers  (2023-07-04T07:53:55Z)
• LORE: Logical Location Regression Network for Table Structure Recognition [24.45544796305824]
  Table structure recognition aims at extracting tables in images into machine-understandable formats. Recent methods solve this problem by predicting the adjacency relations of detected cell boxes. We propose a new TSR framework called LORE, standing for LOgical location REgression network.
  arXiv  Detail & Related papers  (2023-03-07T08:42:46Z)
• Equivariance with Learned Canonicalization Functions [77.32483958400282]
  We show that learning a small neural network to perform canonicalization is better than using predefineds. Our experiments show that learning the canonicalization function is competitive with existing techniques for learning equivariant functions across many tasks.
  arXiv  Detail & Related papers  (2022-11-11T21:58:15Z)
• Reinforcement Learning Based Query Vertex Ordering Model for Subgraph Matching [58.39970828272366]
  Subgraph matching algorithms enumerate all is embeddings of a query graph in a data graph G. matching order plays a critical role in time efficiency of these backtracking based subgraph matching algorithms. In this paper, for the first time we apply the Reinforcement Learning (RL) and Graph Neural Networks (GNNs) techniques to generate the high-quality matching order for subgraph matching algorithms.
  arXiv  Detail & Related papers  (2022-01-25T00:10:03Z)
• Multi-Step Inference for Reasoning Over Paragraphs [95.91527524872832]
  Complex reasoning over text requires understanding and chaining together free-form predicates and logical connectives. We present a compositional model reminiscent of neural module networks that can perform chained logical reasoning.
  arXiv  Detail & Related papers  (2020-04-06T21:12:53Z)
• Anchor & Transform: Learning Sparse Embeddings for Large Vocabularies [60.285091454321055]
  We design a simple and efficient embedding algorithm that learns a small set of anchor embeddings and a sparse transformation matrix. On text classification, language modeling, and movie recommendation benchmarks, we show that ANT is particularly suitable for large vocabulary sizes.
  arXiv  Detail & Related papers  (2020-03-18T13:07:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.