Building Concise Logical Patterns by Constraining Tsetlin Machine Clause Size

• URL: http://arxiv.org/abs/2301.08190v1
• Date: Thu, 19 Jan 2023 17:37:48 GMT
• Title: Building Concise Logical Patterns by Constraining Tsetlin Machine Clause Size
• Authors: K. Darshana Abeyrathna and Ahmed Abdulrahem Othman Abouzeid and Bimal Bhattarai and Charul Giri and Sondre Glimsdal and Ole-Christoffer Granmo and Lei Jiao and Rupsa Saha and Jivitesh Sharma and Svein Anders Tunheim and Xuan Zhang
• Abstract summary: This paper introduces a novel variant of TM learning - Clause Size Constrained TMs (CSC-TMs) As soon as a clause includes more literals than the constraint allows, it starts expelling literals. Our results show that CSC-TM maintains accuracy with up to 80 times fewer literals.
• Score: 11.43224924974832
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Tsetlin machine (TM) is a logic-based machine learning approach with the crucial advantages of being transparent and hardware-friendly. While TMs match or surpass deep learning accuracy for an increasing number of applications, large clause pools tend to produce clauses with many literals (long clauses). As such, they become less interpretable. Further, longer clauses increase the switching activity of the clause logic in hardware, consuming more power. This paper introduces a novel variant of TM learning - Clause Size Constrained TMs (CSC-TMs) - where one can set a soft constraint on the clause size. As soon as a clause includes more literals than the constraint allows, it starts expelling literals. Accordingly, oversized clauses only appear transiently. To evaluate CSC-TM, we conduct classification, clustering, and regression experiments on tabular data, natural language text, images, and board games. Our results show that CSC-TM maintains accuracy with up to 80 times fewer literals. Indeed, the accuracy increases with shorter clauses for TREC, IMDb, and BBC Sports. After the accuracy peaks, it drops gracefully as the clause size approaches a single literal. We finally analyze CSC-TM power consumption and derive new convergence properties.

Related papers

• Hierarchical Context Merging: Better Long Context Understanding for Pre-trained LLMs [61.40047491337793]
  We present Hierarchical cOntext MERging (HOMER), a new training-free scheme designed to overcome the limitations of large language models. HomeR uses a divide-and-conquer algorithm, dividing long inputs into manageable chunks. A token reduction technique precedes each merging, ensuring memory usage efficiency.
  arXiv  Detail & Related papers  (2024-04-16T06:34:08Z)
• Leveraging Denoised Abstract Meaning Representation for Grammatical Error Correction [53.55440811942249]
  Grammatical Error Correction (GEC) is the task of correcting errorful sentences into grammatically correct, semantically consistent, and coherent sentences. We propose the AMR-GEC, a seq-to-seq model that incorporates denoised AMR as additional knowledge.
  arXiv  Detail & Related papers  (2023-07-05T09:06:56Z)
• CTC-based Non-autoregressive Speech Translation [51.37920141751813]
  We investigate the potential of connectionist temporal classification for non-autoregressive speech translation. We develop a model consisting of two encoders that are guided by CTC to predict the source and target texts. Experiments on the MuST-C benchmarks show that our NAST model achieves an average BLEU score of 29.5 with a speed-up of 5.67$times$.
  arXiv  Detail & Related papers  (2023-05-27T03:54:09Z)
• CTC Alignments Improve Autoregressive Translation [145.90587287444976]
  We argue that CTC does in fact make sense for translation if applied in a joint CTC/attention framework. Our proposed joint CTC/attention models outperform pure-attention baselines across six benchmark translation tasks.
  arXiv  Detail & Related papers  (2022-10-11T07:13:50Z)
• An Experimental Study of Permanently Stored Learned Clauses [0.0]
  We study the permanent clause store in MapleLCMDistChronoBT. We show that alternate size and LBD based criteria improve performance, while still having large permanent stores.
  arXiv  Detail & Related papers  (2021-10-27T05:36:16Z)
• CTC Variations Through New WFST Topologies [79.94035631317395]
  This paper presents novel Weighted Finite-State Transducer (WFST) topologies to implement Connectionist Temporal Classification (CTC)-like algorithms for automatic speech recognition. Three new CTC variants are proposed: (1) the "compact-CTC", in which direct transitions between units are replaced with epsilon> back-off transitions; (2) the "minimal-CTC", that only adds blank> self-loops when used in WFST-composition; and (3) "selfless-CTC", that disallows self-loop for non-blank units.
  arXiv  Detail & Related papers  (2021-10-06T23:00:15Z)
• Coalesced Multi-Output Tsetlin Machines with Clause Sharing [7.754230120409288]
  Using finite-state machines to learn patterns, Tsetlin machines (TMs) have obtained competitive accuracy and learning speed across several benchmarks. We introduce clause sharing, merging multiple TMs into a single one. Our empirical results on MNIST, Fashion-MNIST, and Kuzushiji-MNIST show that CoTM obtains significantly higher accuracy than TM on $50$- to $1$K-clause configurations.
  arXiv  Detail & Related papers  (2021-08-17T12:52:01Z)
• Benchmarking Multivariate Time Series Classification Algorithms [69.12151492736524]
  Time Series Classification (TSC) involved building predictive models for a discrete target variable from ordered, real valued, attributes. Over recent years, a new set of TSC algorithms have been developed which have made significant improvement over the previous state of the art. We review recently proposed bespoke MTSC algorithms based on deep learning, shapelets and bag of words approaches.
  arXiv  Detail & Related papers  (2020-07-26T15:56:40Z)
• Extending the Tsetlin Machine With Integer-Weighted Clauses for Increased Interpretability [9.432068833600884]
  Building machine models that are both interpretable and accurate is an unresolved challenge for many pattern recognition problems. Using a linear combination of conjunctive clauses in propositional logic, Tsetlin Machines (TMs) have shown competitive performance on diverse benchmarks. Here, we address the accuracy-interpretability challenge by equipping the TM clauses with integer weights.
  arXiv  Detail & Related papers  (2020-05-11T14:18:09Z)
• On the Effect of Learned Clauses on Stochastic Local Search [0.0]
  Conflict-driven clause learning (CDCL) and local search (SLS) are used in SAT solvers. We experimentally demonstrate that clauses with a large number of correct literals are beneficial to the runtime of SLS. We deduce the most beneficial strategies to add high-quality clauses as a preprocessing step.
  arXiv  Detail & Related papers  (2020-05-07T13:33:16Z)
• A Regression Tsetlin Machine with Integer Weighted Clauses for Compact Pattern Representation [9.432068833600884]
  The Regression Tsetlin Machine (RTM) addresses the lack of interpretability impeding state-of-the-art nonlinear regression models. We introduce integer weighted clauses to reduce computation cost N times and increase interpretability. We evaluate the potential of the integer weighted RTM using six artificial datasets.
  arXiv  Detail & Related papers  (2020-02-04T12:06:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.