Expressivity of Shallow Neural Networks Over Finite Fields

• URL: http://arxiv.org/abs/2607.17090v1
• Date: Sun, 19 Jul 2026 06:02:43 GMT
• Title: Expressivity of Shallow Neural Networks Over Finite Fields
• Abstract summary: We study the expressivity of shallow neural networks (PNNs) with monomial activation functions over finite fields.<n>We quantify the expressivity by the cardinality of the neuromanifold and derive a natural lower and upper bound.<n>We present an architecture that exhibits a striking difference in the neuromanifolds when considered over a characteristic zero versus a finite-characteristic field.
• Score: 5.181088692457079
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We study the expressivity of shallow polynomial neural networks (PNNs) with monomial activation functions over finite fields. For a given architecture, we define a neuromanifold as the image of the map from all possible network weights into the product of polynomial rings. We quantify the expressivity by the cardinality of the neuromanifold, and derive a natural lower and upper bound. This leads to counting rational points over finite fields, a problem closely linked to the Weil conjectures. Finally, we present an architecture that exhibits a striking difference in the neuromanifolds when considered over a characteristic zero versus a finite-characteristic field, illustrating the critical role of field characteristic in the notion of expressivity.

Related papers

• AffineLens: Capturing the Continuous Piecewise Affine Functions of Neural Networks [23.025211488990077]
  Piecewise affine neural networks (PANNs) provide a principled geometric perspective on neural network expressivity.<n>We present AffineLens, a unified framework for computing the hyperplane arrangements and polyhedral structures underlying PANNs.
  arXiv  Detail & Related papers  (2026-05-07T13:16:39Z)
• A Unified Representation of Neural Networks Architectures [2.7018389296091505]
  In this paper we consider the limiting case of neural networks (NNs) architectures when the number of neurons in each hidden layer and the number of hidden layers tend to infinity thus forming a continuum.<n>We show that most of the existing finite and infinite-dimensional NNs are related via homogenization/discretization with the DiPaNet representation.
  arXiv  Detail & Related papers  (2025-12-19T14:01:50Z)
• Activation degree thresholds and expressiveness of polynomial neural networks [0.4666493857924357]
  We study the expressive power of deep neural networks through the geometry of their neurovariety.<n>We prove the existence of the activation degree threshold for all neural networks without the width-one bottlenecks.
  arXiv  Detail & Related papers  (2024-08-08T16:28:56Z)
• Structure of universal formulas [13.794391803767617]
  We introduce a hierarchy of classes connecting the global approximability property to the weaker property of infinite VC dimension. We show that fixed-size neural networks with not more than one layer of neurons having activations cannot approximate functions on arbitrary finite sets. We give examples of functional families, including two-hidden-layer neural networks, that approximate functions on arbitrary finite sets, but fail to do that on the whole domain of definition.
  arXiv  Detail & Related papers  (2023-11-07T11:50:25Z)
• Data Topology-Dependent Upper Bounds of Neural Network Widths [52.58441144171022]
  We first show that a three-layer neural network can be designed to approximate an indicator function over a compact set. This is then extended to a simplicial complex, deriving width upper bounds based on its topological structure. We prove the universal approximation property of three-layer ReLU networks using our topological approach.
  arXiv  Detail & Related papers  (2023-05-25T14:17:15Z)
• On the Expressiveness and Generalization of Hypergraph Neural Networks [77.65788763444877]
  This extended abstract describes a framework for analyzing the expressiveness, learning, and (structural) generalization of hypergraph neural networks (HyperGNNs) Specifically, we focus on how HyperGNNs can learn from finite datasets and generalize structurally to graph reasoning problems of arbitrary input sizes.
  arXiv  Detail & Related papers  (2023-03-09T18:42:18Z)
• Polynomial Neural Fields for Subband Decomposition and Manipulation [78.2401411189246]
  We propose a new class of neural fields called neural fields (PNFs) The key advantage of a PNF is that it can represent a signal as a composition of manipulable and interpretable components without losing the merits of neural fields. We empirically demonstrate that Fourier PNFs enable signal manipulation applications such as texture transfer and scale-space.
  arXiv  Detail & Related papers  (2023-02-09T18:59:04Z)
• Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Study [55.12108376616355]
  The study on NTK has been devoted to typical neural network architectures, but is incomplete for neural networks with Hadamard products (NNs-Hp) In this work, we derive the finite-width-K formulation for a special class of NNs-Hp, i.e., neural networks. We prove their equivalence to the kernel regression predictor with the associated NTK, which expands the application scope of NTK.
  arXiv  Detail & Related papers  (2022-09-16T06:36:06Z)
• Universal approximation property of invertible neural networks [76.95927093274392]
  Invertible neural networks (INNs) are neural network architectures with invertibility by design. Thanks to their invertibility and the tractability of Jacobian, INNs have various machine learning applications such as probabilistic modeling, generative modeling, and representation learning.
  arXiv  Detail & Related papers  (2022-04-15T10:45:26Z)
• Intrinsic Neural Fields: Learning Functions on Manifolds [45.269698580847916]
  Intrinsic neural fields combine the advantages of neural fields with the spectral properties of the Laplace-Beltrami operator. We show that intrinsic neural fields can reconstruct high-quality textures from images with state-of-the-art quality. We demonstrate the versatility of intrinsic neural fields by tackling various applications.
  arXiv  Detail & Related papers  (2022-03-15T14:52:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.