TMComposites: Plug-and-Play Collaboration Between Specialized Tsetlin Machines

• URL: http://arxiv.org/abs/2309.04801v2
• Date: Tue, 12 Sep 2023 15:00:36 GMT
• Title: TMComposites: Plug-and-Play Collaboration Between Specialized Tsetlin Machines
• Authors: Ole-Christoffer Granmo
• Abstract summary: This paper introduces plug-and-play collaboration between specialized TMs, referred to as TM Composites. The collaboration relies on a TM's ability to specialize during learning and to assess its competence during inference. We implement three TM specializations in our empirical evaluation.
• Score: 12.838678214659422
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Tsetlin Machines (TMs) provide a fundamental shift from arithmetic-based to logic-based machine learning. Supporting convolution, they deal successfully with image classification datasets like MNIST, Fashion-MNIST, and CIFAR-2. However, the TM struggles with getting state-of-the-art performance on CIFAR-10 and CIFAR-100, representing more complex tasks. This paper introduces plug-and-play collaboration between specialized TMs, referred to as TM Composites. The collaboration relies on a TM's ability to specialize during learning and to assess its competence during inference. When teaming up, the most confident TMs make the decisions, relieving the uncertain ones. In this manner, a TM Composite becomes more competent than its members, benefiting from their specializations. The collaboration is plug-and-play in that members can be combined in any way, at any time, without fine-tuning. We implement three TM specializations in our empirical evaluation: Histogram of Gradients, Adaptive Gaussian Thresholding, and Color Thermometers. The resulting TM Composite increases accuracy on Fashion-MNIST by two percentage points, CIFAR-10 by twelve points, and CIFAR-100 by nine points, yielding new state-of-the-art results for TMs. Overall, we envision that TM Composites will enable an ultra-low energy and transparent alternative to state-of-the-art deep learning on more tasks and datasets.

Related papers

• Code and Pixels: Multi-Modal Contrastive Pre-training for Enhanced Tabular Data Analysis [3.640521552987694]
  We present Multi-task Contrastive Masked Tabular Modeling (MT-CMTM) We employ a dual strategy combining contrastive learning with masked tabular modeling, optimizing the synergy between these data modalities. Central to our approach is a 1D Convolutional Neural Network with residual connections and an attention mechanism (1D-ResNet-CBAM)
  arXiv  Detail & Related papers  (2025-01-13T13:12:18Z)
• MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts [63.67734699877724]
  MoE++ is a general and heterogeneous MoE framework that integrates both Feed-Forward Network(FFN) and zero-computation experts. MoE++ achieves better performance while delivering 1.1-2.1x expert forward throughput compared to a vanilla MoE model of the same size.
  arXiv  Detail & Related papers  (2024-10-09T18:01:27Z)
• MC-MoE: Mixture Compressor for Mixture-of-Experts LLMs Gains More [71.0473038084673]
  We propose MC-MoE, a training-free Mixture-Compressor for Mixture-of-Experts large language models (MoE-LLMs) MC-MoE leverages the significance of both experts and tokens to achieve an extreme compression. For instance, at 2.54 bits, MC-MoE compresses 76.6% of the model, with only a 3.8% average accuracy loss.
  arXiv  Detail & Related papers  (2024-10-08T18:09:38Z)
• PMT-MAE: Dual-Branch Self-Supervised Learning with Distillation for Efficient Point Cloud Classification [46.266960248570086]
  This paper introduces PMT-MAE, a novel self-supervised learning framework for point cloud classification. PMT-MAE features a dual-branch architecture that integrates Transformer and components to capture rich features. PMT-MAE surpasses the baseline Point-MAE (93.2%) and the teacher Point-M2AE (93.4%), underscoring its ability to learn discnative 3D point cloud representations.
  arXiv  Detail & Related papers  (2024-09-03T15:54:34Z)
• Multi-Modality Co-Learning for Efficient Skeleton-based Action Recognition [12.382193259575805]
  We propose a novel multi-modality co-learning (MMCL) framework for efficient skeleton-based action recognition. Our MMCL framework engages in multi-modality co-learning during the training stage and keeps efficiency by employing only concise skeletons in inference.
  arXiv  Detail & Related papers  (2024-07-22T15:16:47Z)
• SAM as the Guide: Mastering Pseudo-Label Refinement in Semi-Supervised Referring Expression Segmentation [66.92696817276288]
  SemiRES is a semi-supervised framework that effectively leverages a combination of labeled and unlabeled data to perform RES. SemiRES incorporates the Segment Anything Model (SAM), renowned for its precise boundary demarcation. In instances where a precise mask cannot be matched from the available candidates, we develop the Pixel-Wise Adjustment (PWA) strategy.
  arXiv  Detail & Related papers  (2024-06-03T15:42:30Z)
• An Optimized Toolbox for Advanced Image Processing with Tsetlin Machine Composites [9.669581902728552]
  We leverage the recently proposed TM Composites architecture and introduce a range of TM Specialists. These include Canny edge detection, Histogram of Oriented Gradients, adaptive mean thresholding, adaptive Gaussian thresholding, Otsu's thresholding, color thermometers, and adaptive color thermometers. The result is a toolbox that provides new state-of-the-art results on CIFAR-10 for TMs with an accuracy of 82.8%.
  arXiv  Detail & Related papers  (2024-06-02T10:52:48Z)
• A Contrastive Compositional Benchmark for Text-to-Image Synthesis: A Study with Unified Text-to-Image Fidelity Metrics [58.83242220266935]
  We introduce Winoground-T2I, a benchmark designed to evaluate the compositionality of T2I models. This benchmark includes 11K complex, high-quality contrastive sentence pairs spanning 20 categories. We use Winoground-T2I with a dual objective: to evaluate the performance of T2I models and the metrics used for their evaluation.
  arXiv  Detail & Related papers  (2023-12-04T20:47:48Z)
• On the Equivalence of the Weighted Tsetlin Machine and the Perceptron [12.48513712803069]
  Tsetlin Machine (TM) has been gaining popularity as an inherently interpretable machine leaning method. Although possessing favorable properties, TM has not been the go-to method for AI applications.
  arXiv  Detail & Related papers  (2022-12-27T22:38:59Z)
• Multi-scale Transformer Network with Edge-aware Pre-training for Cross-Modality MR Image Synthesis [52.41439725865149]
  Cross-modality magnetic resonance (MR) image synthesis can be used to generate missing modalities from given ones. Existing (supervised learning) methods often require a large number of paired multi-modal data to train an effective synthesis model. We propose a Multi-scale Transformer Network (MT-Net) with edge-aware pre-training for cross-modality MR image synthesis.
  arXiv  Detail & Related papers  (2022-12-02T11:40:40Z)
• FedGradNorm: Personalized Federated Gradient-Normalized Multi-Task Learning [50.756991828015316]
  Multi-task learning (MTL) is a novel framework to learn several tasks simultaneously with a single shared network. We propose FedGradNorm which uses a dynamic-weighting method to normalize norms in order to balance learning speeds among different tasks.
  arXiv  Detail & Related papers  (2022-03-24T17:43:12Z)
• Transductive Maximum Margin Classifier for Few-Shot Learning [17.18278071760926]
  Few-shot learning aims to train a classifier that can generalize well when just a small number of labeled samples per class are given. We introduce Transductive Maximum Margin (TMMC) for corresponding few-shot learning. TMMC is constructed using a mixture of the labeled support set and the unlabeled query set in a given task.
  arXiv  Detail & Related papers  (2021-07-26T06:02:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.