Learning with Volterra Neural Networks: A System Theoretic Perspective

• URL: http://arxiv.org/abs/2609.01928v1
• Date: Tue, 01 Sep 2026 22:58:05 GMT
• Title: Learning with Volterra Neural Networks: A System Theoretic Perspective
• Abstract summary: kVNN is a learnable kernelized Volterra Neural operator for compact higher-order filtering.<n>The proposed formulation combines the order-wise structure of Volterra filtering with learnable atoms, allowing different interaction orders to be represented.
• Score: 6.550684951976901
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Higher-order interaction components are important for signal, image, and video modeling, but explicit high-order operators often suffer from rapidly increasing parameter and computational costs. This paper presents kVNN, a learnable kernelized Volterra Neural operator for compact higher-order filtering. The motivation is to use kernelization to improve the efficiency of Volterra-type neural operators while providing a structured interpretation of their higher-order components. The proposed formulation combines the order-wise structure of Volterra filtering with learnable polynomial-kernel atoms, allowing different interaction orders to be represented by separate learnable centers and coefficients. This order-decoupled representation avoids explicit high-order tensor parameterization and can be implemented as a CNN-compatible layer. Experiments on representative vision tasks show that kVNN achieves a favorable accuracy--efficiency trade-off.

Related papers

• KVNN: Learnable Multi-Kernel Volterra Neural Networks [6.550684951976901]
  kernelized Volterra Neural Network (kVNN)<n>Experiments on two representative tasks: video action recognition and image denoising.<n>Results are maintained even when trained from scratch without large-scale pretraining.
  arXiv  Detail & Related papers  (2026-04-16T15:18:48Z)
• Spectral Higher-Order Neural Networks [0.6999740786886536]
  We present Spectral Higher-Order Neural Networks (SHONNs), a new algorithmic strategy to incorporate higher-order interactions in general-purpose, feedforward, network structures.<n>This allows to mitigate the common stability and parameter scaling problems that come along weighted, higher-order, forward propagations.
  arXiv  Detail & Related papers  (2026-03-30T13:29:34Z)
• Signal Processing for Implicit Neural Representations [80.38097216996164]
  Implicit Neural Representations (INRs) encode continuous multi-media data via multi-layer perceptrons. Existing works manipulate such continuous representations via processing on their discretized instance. We propose an implicit neural signal processing network, dubbed INSP-Net, via differential operators on INR.
  arXiv  Detail & Related papers  (2022-10-17T06:29:07Z)
• Deep Architecture Connectivity Matters for Its Convergence: A Fine-Grained Analysis [94.64007376939735]
  We theoretically characterize the impact of connectivity patterns on the convergence of deep neural networks (DNNs) under gradient descent training. We show that by a simple filtration on "unpromising" connectivity patterns, we can trim down the number of models to evaluate.
  arXiv  Detail & Related papers  (2022-05-11T17:43:54Z)
• Dynamic Inference with Neural Interpreters [72.90231306252007]
  We present Neural Interpreters, an architecture that factorizes inference in a self-attention network as a system of modules. inputs to the model are routed through a sequence of functions in a way that is end-to-end learned. We show that Neural Interpreters perform on par with the vision transformer using fewer parameters, while being transferrable to a new task in a sample efficient manner.
  arXiv  Detail & Related papers  (2021-10-12T23:22:45Z)
• Optimising for Interpretability: Convolutional Dynamic Alignment Networks [108.83345790813445]
  We introduce a new family of neural network models called Convolutional Dynamic Alignment Networks (CoDA Nets) Their core building blocks are Dynamic Alignment Units (DAUs), which are optimised to transform their inputs with dynamically computed weight vectors that align with task-relevant patterns. CoDA Nets model the classification prediction through a series of input-dependent linear transformations, allowing for linear decomposition of the output into individual input contributions.
  arXiv  Detail & Related papers  (2021-09-27T12:39:46Z)
• Latent Code-Based Fusion: A Volterra Neural Network Approach [21.25021807184103]
  We propose a deep structure encoder using the recently introduced Volterra Neural Networks (VNNs) We show that the proposed approach demonstrates a much-improved sample complexity over CNN-based auto-encoder with a superb robust classification performance.
  arXiv  Detail & Related papers  (2021-04-10T18:29:01Z)
• AdnFM: An Attentive DenseNet based Factorization Machine for CTR Prediction [11.958336595818267]
  We propose a novel model called Attentive DenseNet based Factorization Machines (AdnFM) AdnFM can extract more comprehensive deep features by using all the hidden layers from a feed-forward neural network as implicit high-order features. Experiments on two real-world datasets show that the proposed model can effectively improve the performance of Click-Through-Rate prediction.
  arXiv  Detail & Related papers  (2020-12-20T01:00:39Z)
• ACDC: Weight Sharing in Atom-Coefficient Decomposed Convolution [57.635467829558664]
  We introduce a structural regularization across convolutional kernels in a CNN. We show that CNNs now maintain performance with dramatic reduction in parameters and computations.
  arXiv  Detail & Related papers  (2020-09-04T20:41:47Z)
• Supervised Learning for Non-Sequential Data: A Canonical Polyadic Decomposition Approach [85.12934750565971]
  Efficient modelling of feature interactions underpins supervised learning for non-sequential tasks. To alleviate this issue, it has been proposed to implicitly represent the model parameters as a tensor. For enhanced expressiveness, we generalize the framework to allow feature mapping to arbitrarily high-dimensional feature vectors.
  arXiv  Detail & Related papers  (2020-01-27T22:38:40Z)
• Volterra Neural Networks (VNNs) [24.12314339259243]
  We propose a Volterra filter-inspired Network architecture to reduce the complexity of Convolutional Neural Networks. We show an efficient parallel implementation of this Volterra Neural Network (VNN) along with its remarkable performance. The proposed approach is evaluated on UCF-101 and HMDB-51 datasets for action recognition, and is shown to outperform state of the art CNN approaches.
  arXiv  Detail & Related papers  (2019-10-21T19:22:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.