Tensor-Augmented Convolutional Neural Networks: Enhancing Expressivity with Generic Tensor Kernels

• URL: http://arxiv.org/abs/2604.08072v1
• Date: Thu, 09 Apr 2026 10:44:23 GMT
• Title: Tensor-Augmented Convolutional Neural Networks: Enhancing Expressivity with Generic Tensor Kernels
• Abstract summary: Convolutional Neural Networks (CNNs) excel at extracting local features hierarchically, but their performance in capturing complex correlations hinges heavily on deep architectures.<n>We propose a physically-guided shallow model: tensor-augmented CNN (TACNN)<n>In our design the convolution output of each layer becomes a multilinear form capable of capturing high-order feature correlations.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Convolutional Neural Networks (CNNs) excel at extracting local features hierarchically, but their performance in capturing complex correlations hinges heavily on deep architectures, which are usually computationally demanding and difficult to interpret. To address these issues, we propose a physically-guided shallow model: tensor-augmented CNN (TACNN), which replaces conventional convolution kernels with generic tensors to enhance representational capacity. This choice is motivated by the fact that an order-$N$ tensor naturally encodes an arbitrary quantum superposition state in the Hilbert space of dimension $d^N$, where $d$ is the local physical dimension, thus offering substantially richer expressivity. Furthermore, in our design the convolution output of each layer becomes a multilinear form capable of capturing high-order feature correlations, thereby equipping a shallow multilayer architecture with an expressive power competitive to that of deep CNNs. On the Fashion-MNIST benchmark, TACNN demonstrates clear advantages over conventional CNNs, achieving remarkable accuracies with only a few layers. In particular, a TACNN with only two convolution layers attains a test accuracy of 93.7$\%$, surpassing or matching considerably deeper models such as VGG-16 (93.5$\%$) and GoogLeNet (93.7$\%$). These findings highlight TACNN as a promising framework that strengthens model expressivity while preserving architectural simplicity, paving the way towards more interpretable and efficient deep learning models.

Related papers

• Reducing ANN-SNN Conversion Error via Residual Membrane Potential Alignment [52.910770415431244]
  Spiking Neural Networks (SNNs) serve as core architectures for neuromorphic computing.<n>ANN-SNN conversion circumvents issues by reusing well-trained ANN weights for low-latency, energy-efficient inference.<n>Existing conversion schemes suffer from severe accuracy drops at small timesteps, large inference delays and cumulative quantization errors.
  arXiv  Detail & Related papers  (2026-08-14T04:46:03Z)
• Differentiable Lifting for Topological Neural Networks [49.208571794040374]
  Topological neural networks (TNNs) enable leveraging high-order structures on graphs.<n>DiffLift is a framework for learning graph liftings to hypergraphs and cellular- and simplicial complexes.<n>Our approach leads to gains of up to 45% over static liftings, including both connectivity- and feature-based ones.
  arXiv  Detail & Related papers  (2026-08-02T11:30:09Z)
• Kernel Renormalization in Bayesian Deep Neural Networks: the Equivalent Wishart Ansatz in the Proportional Regime [33.80781505782195]
  We present an effective approximate approach to predict the performance of multi-layer perceptrons (MLPs) of fixed depth $L$ on arbitrary high-dimensional data.<n>Even strong representation learning in the proportional limit is encoded in at most $L$ order parameters, determined self-consistently.
  arXiv  Detail & Related papers  (2026-05-28T09:49:54Z)
• M$^2$RNN: Non-Linear RNNs with Matrix-Valued States for Scalable Language Modeling [53.351103753419046]
  We introduce Matrix-to-Matrix RNN: an architecture with matrix-valued hidden states and expressive non-linear state transitions.<n>We demonstrate that the language modeling performance of non-linear RNNs is limited by their state size.<n>We also demonstrate how the state size expansion mechanism enables efficient use of tensor cores.
  arXiv  Detail & Related papers  (2026-03-15T12:53:09Z)
• S$^2$NN: Sub-bit Spiking Neural Networks [53.08060832135342]
  Spiking Neural Networks (SNNs) offer an energy-efficient paradigm for machine intelligence.<n>Despite recent advances in binary SNNs, the storage and computational demands remain substantial for large-scale networks.<n>We propose Sub-bit Spiking Neural Networks (S$2$NNs) that represent weights with less than one bit.
  arXiv  Detail & Related papers  (2025-09-29T04:17:44Z)
• Convexified Message-Passing Graph Neural Networks [12.350115354262947]
  We introduce Convexified Message Passing Graph Neural Networks (CGNNs)<n>By mapping their nonlinear Hilbert kernel training into a convex optimization problem, CGNNs transform into a convex optimization problem.<n>Experiments benchmark datasets show that CGNNs significantly exceed the performance of leading GNN models.
  arXiv  Detail & Related papers  (2025-05-23T18:33:01Z)
• ScaleGNN: Towards Scalable Graph Neural Networks via Adaptive High-order Neighboring Feature Fusion [73.85920403511706]
  We propose ScaleGNN, a novel framework that adaptively fuses multi-hop node features for scalable and effective graph learning.<n>We show that ScaleGNN consistently outperforms state-of-the-art GNNs in both predictive accuracy and computational efficiency.
  arXiv  Detail & Related papers  (2025-04-22T14:05:11Z)
• Compositionality Unlocks Deep Interpretable Models [0.5437298646956507]
  $chi$-net is an intrinsically interpretable architecture combining the compositional multilinear structure of tensor networks with the expressivity and efficiency of deep neural networks.
  arXiv  Detail & Related papers  (2025-04-03T15:07:54Z)
• Unveiling the Unseen: Identifiable Clusters in Trained Depthwise Convolutional Kernels [56.69755544814834]
  Recent advances in depthwise-separable convolutional neural networks (DS-CNNs) have led to novel architectures. This paper reveals another striking property of DS-CNN architectures: discernible and explainable patterns emerge in their trained depthwise convolutional kernels in all layers.
  arXiv  Detail & Related papers  (2024-01-25T19:05:53Z)
• MuseGNN: Forming Scalable, Convergent GNN Layers that Minimize a Sampling-Based Energy [27.328628715885287]
  A graph neural network (GNN) architecture is capable of modeling data with cross-instance relations.<n>An important subclass of this architecture involves layers designed such that the forward pass iteratively reduces a graph-regularized energy function of interest.<n>We propose a sampling-based energy function and scalable GNN layers that iteratively reduce it, guided by convergence guarantees in certain settings.<n>We also instantiate a full GNN architecture based on these designs, and the model achieves competitive accuracy and scalability when applied to the largest publicly-available node classification benchmark exceeding 1TB in size.
  arXiv  Detail & Related papers  (2023-10-19T04:30:14Z)
• Contextualizing MLP-Mixers Spatiotemporally for Urban Data Forecast at Scale [54.15522908057831]
  We propose an adapted version of the computationally-Mixer for STTD forecast at scale. Our results surprisingly show that this simple-yeteffective solution can rival SOTA baselines when tested on several traffic benchmarks. Our findings contribute to the exploration of simple-yet-effective models for real-world STTD forecasting.
  arXiv  Detail & Related papers  (2023-07-04T05:19:19Z)
• Deep Architecture Connectivity Matters for Its Convergence: A Fine-Grained Analysis [94.64007376939735]
  We theoretically characterize the impact of connectivity patterns on the convergence of deep neural networks (DNNs) under gradient descent training. We show that by a simple filtration on "unpromising" connectivity patterns, we can trim down the number of models to evaluate.
  arXiv  Detail & Related papers  (2022-05-11T17:43:54Z)
• Block-term Tensor Neural Networks [29.442026567710435]
  We show that block-term tensor layers (BT-layers) can be easily adapted to neural network models, such as CNNs and RNNs. BT-layers in CNNs and RNNs can achieve a very large compression ratio on the number of parameters while preserving or improving the representation power of the original DNNs.
  arXiv  Detail & Related papers  (2020-10-10T09:58:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.