Directional Non-Commutative Monoidal Embeddings for MNIST

• URL: http://arxiv.org/abs/2506.03472v1
• Date: Wed, 04 Jun 2025 01:07:51 GMT
• Title: Directional Non-Commutative Monoidal Embeddings for MNIST
• Authors: Mahesh Godavarti,
• Abstract summary: We present an empirical validation of the directional non-commutative monoidal embedding framework.<n>This framework defines learnable compositional embeddings using distinct non-commutative operators per dimension.<n>Our experiments confirm that directional non-commutative monoidal embeddings are highly effective for representing image data.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We present an empirical validation of the directional non-commutative monoidal embedding framework recently introduced in prior work~\cite{Godavarti2025monoidal}. This framework defines learnable compositional embeddings using distinct non-commutative operators per dimension (axis) that satisfy an interchange law, generalizing classical one-dimensional transforms. Our primary goal is to verify that this framework can effectively model real data by applying it to a controlled, well-understood task: image classification on the MNIST dataset~\cite{lecun1998gradient}. A central hypothesis for why the proposed monoidal embedding works well is that it generalizes the Discrete Fourier Transform (DFT)~\cite{oppenheim1999discrete} by learning task-specific frequency components instead of using fixed basis frequencies. We test this hypothesis by comparing learned monoidal embeddings against fixed DFT-based embeddings on MNIST. The results show that as the embedding dimensionality decreases (e.g., from 32 to 8 to 2), the performance gap between the learned monoidal embeddings and fixed DFT-based embeddings on MNIST grows increasingly large. This comparison is used as an analytic tool to explain why the framework performs well: the learnable embeddings can capture the most discriminative spectral components for the task. Overall, our experiments confirm that directional non-commutative monoidal embeddings are highly effective for representing image data, offering a compact learned representation that retains high task performance. The code used in this work is available at https://github.com/mahesh-godavarti/directional_composition_mnist.

Related papers

• How Learnable Grids Recover Fine Detail in Low Dimensions: A Neural Tangent Kernel Analysis of Multigrid Parametric Encodings [106.3726679697804]
  We compare the two most common techniques for mitigating this spectral bias: Fourier feature encodings (FFE) and multigrid parametric encodings (MPE)<n>MPEs are seen as the standard for low dimensional mappings, but MPEs often outperform them and learn representations with higher resolution and finer detail.<n>We prove that MPEs improve a network's performance through the structure of their grid and not their learnable embedding.
  arXiv  Detail & Related papers  (2025-04-18T02:18:08Z)
• Relaxed Rotational Equivariance via $G$-Biases in Vision [19.814324876189772]
  Group Equivariant Convolution (GConv) can capture rotational equivariance from original data.<n>However, the presentation or distribution of real-world data rarely conforms to strict rotational equivariance.<n>We propose a simple but highly effective method to address this problem, which utilizes a set of learnable biases called $G$-Biases.<n> Experiments demonstrate that the proposed RREConv-based methods achieve excellent performance compared to existing GConv-based methods in both classification and 2D object detection tasks.
  arXiv  Detail & Related papers  (2024-08-22T14:52:53Z)
• Fast and interpretable Support Vector Classification based on the truncated ANOVA decomposition [0.0]
  Support Vector Machines (SVMs) are an important tool for performing classification on scattered data. We propose solving SVMs in primal form using feature maps based on trigonometric functions or wavelets.
  arXiv  Detail & Related papers  (2024-02-04T10:27:42Z)
• Uncovering Model Processing Strategies with Non-Negative Per-Example Fisher Factorization [44.39409273712917]
  We introduce NPEFF, an interpretability method that aims to uncover strategies used by a model to generate its predictions.<n>We demonstrate that NPEFF components correspond to model processing strategies for a variety of language models and text processing tasks.
  arXiv  Detail & Related papers  (2023-10-07T02:02:45Z)
• Manifold Learning with Sparse Regularised Optimal Transport [1.949927790632678]
  Real-world datasets are subject to noisy observations and sampling, so that distilling information about the underlying manifold is a major challenge.<n>We propose a method for manifold learning that utilises a symmetric version of optimal transport with a quadratic regularisation.<n>We prove that the resulting kernel is consistent with a Laplace-type operator in the continuous limit, establish robustness to heteroskedastic noise and exhibit these results in numerical experiments.
  arXiv  Detail & Related papers  (2023-07-19T08:05:46Z)
• Disentanglement via Latent Quantization [60.37109712033694]
  In this work, we construct an inductive bias towards encoding to and decoding from an organized latent space. We demonstrate the broad applicability of this approach by adding it to both basic data-re (vanilla autoencoder) and latent-reconstructing (InfoGAN) generative models.
  arXiv  Detail & Related papers  (2023-05-28T06:30:29Z)
• Functional Indirection Neural Estimator for Better Out-of-distribution Generalization [27.291114360472243]
  FINE (Functional Indirection Neural Estorimator) learns to compose functions that map data input to output on-the-fly. We train FINE and competing models on IQ tasks using images from the MNIST, Omniglot and CIFAR100 datasets. FINE not only achieves the best performance on all tasks but also is able to adapt to small-scale data scenarios.
  arXiv  Detail & Related papers  (2022-10-23T14:43:02Z)
• Combating Mode Collapse in GANs via Manifold Entropy Estimation [70.06639443446545]
  Generative Adversarial Networks (GANs) have shown compelling results in various tasks and applications. We propose a novel training pipeline to address the mode collapse issue of GANs.
  arXiv  Detail & Related papers  (2022-08-25T12:33:31Z)
• Improving the Sample-Complexity of Deep Classification Networks with Invariant Integration [77.99182201815763]
  Leveraging prior knowledge on intraclass variance due to transformations is a powerful method to improve the sample complexity of deep neural networks. We propose a novel monomial selection algorithm based on pruning methods to allow an application to more complex problems. We demonstrate the improved sample complexity on the Rotated-MNIST, SVHN and CIFAR-10 datasets.
  arXiv  Detail & Related papers  (2022-02-08T16:16:11Z)
• Efficient Multidimensional Functional Data Analysis Using Marginal Product Basis Systems [2.4554686192257424]
  We propose a framework for learning continuous representations from a sample of multidimensional functional data. We show that the resulting estimation problem can be solved efficiently by the tensor decomposition. We conclude with a real data application in neuroimaging.
  arXiv  Detail & Related papers  (2021-07-30T16:02:15Z)
• Efficient Semi-Implicit Variational Inference [65.07058307271329]
  We propose an efficient and scalable semi-implicit extrapolational (SIVI) Our method maps SIVI's evidence to a rigorous inference of lower gradient values.
  arXiv  Detail & Related papers  (2021-01-15T11:39:09Z)
• Hierarchical Dynamic Filtering Network for RGB-D Salient Object Detection [91.43066633305662]
  The main purpose of RGB-D salient object detection (SOD) is how to better integrate and utilize cross-modal fusion information. In this paper, we explore these issues from a new perspective. We implement a kind of more flexible and efficient multi-scale cross-modal feature processing.
  arXiv  Detail & Related papers  (2020-07-13T07:59:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.