Hyperdimensional Vector Tsetlin Machines with Applications to Sequence Learning and Generation

• URL: http://arxiv.org/abs/2408.16620v1
• Date: Thu, 29 Aug 2024 15:28:01 GMT
• Title: Hyperdimensional Vector Tsetlin Machines with Applications to Sequence Learning and Generation
• Authors: Christian D. Blakely,
• Abstract summary: We construct a two-layered model for learning and generating sequential data that is both computationally fast and competitive with vanilla Tsetlin machines. We apply the approach in two areas, namely in forecasting, generating new sequences, and classification. For the latter, we derive results for the entire UCR Time Series Archive and compare with the standard benchmarks to see how well the method competes in time series classification.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We construct a two-layered model for learning and generating sequential data that is both computationally fast and competitive with vanilla Tsetlin machines, adding numerous advantages. Through the use of hyperdimensional vector computing (HVC) algebras and Tsetlin machine clause structures, we demonstrate that the combination of both inherits the generality of data encoding and decoding of HVC with the fast interpretable nature of Tsetlin machines to yield a powerful machine learning model. We apply the approach in two areas, namely in forecasting, generating new sequences, and classification. For the latter, we derive results for the entire UCR Time Series Archive and compare with the standard benchmarks to see how well the method competes in time series classification.

Related papers

• Blending Low and High-Level Semantics of Time Series for Better Masked Time Series Generation [0.8999666725996975]
  We introduce a novel framework, termed NC-VQVAE, to integrate self-supervised learning into time series generation approaches. Our experimental results demonstrate that NC-VQVAE results in a considerable improvement in the quality of synthetic samples.
  arXiv  Detail & Related papers  (2024-08-29T15:20:17Z)
• CORE: Common Random Reconstruction for Distributed Optimization with Provable Low Communication Complexity [110.50364486645852]
  Communication complexity has become a major bottleneck for speeding up training and scaling up machine numbers. We propose Common Om REOm, which can be used to compress information transmitted between machines.
  arXiv  Detail & Related papers  (2023-09-23T08:45:27Z)
• TSGM: A Flexible Framework for Generative Modeling of Synthetic Time Series [61.436361263605114]
  Time series data are often scarce or highly sensitive, which precludes the sharing of data between researchers and industrial organizations. We introduce Time Series Generative Modeling (TSGM), an open-source framework for the generative modeling of synthetic time series.
  arXiv  Detail & Related papers  (2023-05-19T10:11:21Z)
• A Deep Neural Networks ensemble workflow from hyperparameter search to inference leveraging GPU clusters [0.0]
  AutoML seeks to automatically build ensembles of Deep Neural Networks (DNNs) to achieve qualitative predictions. We propose a new AutoML to build a larger library of accurate and diverse individual models to then construct ensembles. New ensemble selection method based on a multi-objective greedy algorithm is proposed to generate accurate ensembles.
  arXiv  Detail & Related papers  (2022-08-30T08:04:19Z)
• Towards Similarity-Aware Time-Series Classification [51.2400839966489]
  We study time-series classification (TSC), a fundamental task of time-series data mining. We propose Similarity-Aware Time-Series Classification (SimTSC), a framework that models similarity information with graph neural networks (GNNs)
  arXiv  Detail & Related papers  (2022-01-05T02:14:57Z)
• Sketching as a Tool for Understanding and Accelerating Self-attention for Long Sequences [52.6022911513076]
  Transformer-based models are not efficient in processing long sequences due to the quadratic space and time complexity of the self-attention modules. We propose Linformer and Informer to reduce the quadratic complexity to linear (modulo logarithmic factors) via low-dimensional projection and row selection. Based on the theoretical analysis, we propose Skeinformer to accelerate self-attention and further improve the accuracy of matrix approximation to self-attention.
  arXiv  Detail & Related papers  (2021-12-10T06:58:05Z)
• HyperSeed: Unsupervised Learning with Vector Symbolic Architectures [5.258404928739212]
  This paper presents a novel unsupervised machine learning approach named Hyperseed. It leverages Vector Symbolic Architectures (VSA) for fast learning a topology preserving feature map of unlabelled data. The two distinctive novelties of the Hyperseed algorithm are 1) Learning from only few input data samples and 2) A learning rule based on a single vector operation.
  arXiv  Detail & Related papers  (2021-10-15T20:05:43Z)
• Probability-driven scoring functions in combining linear classifiers [0.913755431537592]
  This research is aimed at building a new fusion method dedicated to the ensemble of linear classifiers. The proposed fusion method is compared with the reference method using multiple benchmark datasets taken from the KEEL repository. The experimental study shows that, under certain conditions, some improvement may be obtained.
  arXiv  Detail & Related papers  (2021-09-16T08:58:32Z)
• Few-Shot Named Entity Recognition: A Comprehensive Study [92.40991050806544]
  We investigate three schemes to improve the model generalization ability for few-shot settings. We perform empirical comparisons on 10 public NER datasets with various proportions of labeled data. We create new state-of-the-art results on both few-shot and training-free settings.
  arXiv  Detail & Related papers  (2020-12-29T23:43:16Z)
• Connecting Weighted Automata, Tensor Networks and Recurrent Neural Networks through Spectral Learning [58.14930566993063]
  We present connections between three models used in different research fields: weighted finite automata(WFA) from formal languages and linguistics, recurrent neural networks used in machine learning, and tensor networks. We introduce the first provable learning algorithm for linear 2-RNN defined over sequences of continuous vectors input.
  arXiv  Detail & Related papers  (2020-10-19T15:28:00Z)
• Fast Transformers with Clustered Attention [14.448898156256478]
  We propose clustered attention, which instead of computing the attention for every query, groups queries into clusters and computes attention just for the centroids. This results in a model with linear complexity with respect to the sequence length for a fixed number of clusters. We evaluate our approach on two automatic speech recognition datasets and show that our model consistently outperforms vanilla transformers for a given computational budget.
  arXiv  Detail & Related papers  (2020-07-09T14:17:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.