Efficient Approximation for Encoder--Decoder Neural Operators via Variation Spaces

• URL: http://arxiv.org/abs/2606.01244v1
• Date: Sun, 31 May 2026 13:53:17 GMT
• Title: Efficient Approximation for Encoder--Decoder Neural Operators via Variation Spaces
• Abstract summary: We study operator learning using encoder--decoder neural networks.<n>We introduce a variation space as an infinite-dimensional structural class for nonlinear operators.<n>For operators in this space, we establish approximation bounds for encoder--decoder two-layer networks in the Bochner $Lq$ norm.
• Score: 7.820614736576814
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We study operator learning using encoder--decoder neural networks. Inspired by the function-space theory of neural networks, we introduce a variation space as an infinite-dimensional structural class for nonlinear operators. This space is defined through vector-valued measures directly on the input and output spaces. For operators in this space, we establish approximation bounds for encoder--decoder two-layer networks in the Bochner $L^q$ norm. The resulting error bound decomposes into the input encoding error, the output encoding error, and a finite-width approximation term of order $N^{-1/2}$, with a constant independent of the input and output encoding dimensions. When the input and output encoding errors decay polynomially in the encoding dimensions, these estimates yield algebraic approximation and learning rates. The results provide an theoretical guarantees for efficient neural operator learning beyond general Lipschitz or Fréchet differentiable operator classes.

Related papers

• Resolution-Independent Analysis of Encoder--Decoder Operator Learning via Limiting Kernels [16.10520704144734]
  In encoder-decoder architectures, a matrix-valued kernel induces an operator-valued kernel on the original function spaces.<n>As the input and output resolutions increase, the induced kernels converge to a limiting kernel.
  arXiv  Detail & Related papers  (2026-09-12T08:19:20Z)
• The Von-Neumann State-Space Transformer for neural decoding [0.0]
  Cortical computation is strikingly low-dimensional.<n>We introduce a memory-augmented Transformer whose feed-forward block is a low-rank instruction bank.<n>VN-SST is far more data-efficient than a modern Transformer.
  arXiv  Detail & Related papers  (2026-08-25T19:28:33Z)
• Deep Parallel Spectral Neural Operators for Solving Partial Differential Equations with Enhanced Low-Frequency Learning Capability [11.121415128908566]
  We propose a Deep Parallel Spectral Neural Operator (DPNO) to enhance the ability to learn low-frequency information.<n>Our method enhances the neural operator's ability to learn low-frequency information through parallel modules.<n>We smooth this information through convolutional mappings, thereby reducing high-frequency errors.
  arXiv  Detail & Related papers  (2024-09-30T06:04:04Z)
• Learning Linear Block Error Correction Codes [62.25533750469467]
  We propose for the first time a unified encoder-decoder training of binary linear block codes. We also propose a novel Transformer model in which the self-attention masking is performed in a differentiable fashion for the efficient backpropagation of the code gradient.
  arXiv  Detail & Related papers  (2024-05-07T06:47:12Z)
• Generalization Bounds for Neural Belief Propagation Decoders [10.96453955114324]
  In this paper, we investigate the generalization capabilities of neural network based decoders. Specifically, the generalization gap of a decoder is the difference between empirical and expected bit-error-rate(s) Results are presented for both regular and irregular parity-check matrices.
  arXiv  Detail & Related papers  (2023-05-17T19:56:04Z)
• The END: An Equivariant Neural Decoder for Quantum Error Correction [73.4384623973809]
  We introduce a data efficient neural decoder that exploits the symmetries of the problem. We propose a novel equivariant architecture that achieves state of the art accuracy compared to previous neural decoders.
  arXiv  Detail & Related papers  (2023-04-14T19:46:39Z)
• A Scalable Graph Neural Network Decoder for Short Block Codes [49.25571364253986]
  We propose a novel decoding algorithm for short block codes based on an edge-weighted graph neural network (EW-GNN) The EW-GNN decoder operates on the Tanner graph with an iterative message-passing structure. We show that the EW-GNN decoder outperforms the BP and deep-learning-based BP methods in terms of the decoding error rate.
  arXiv  Detail & Related papers  (2022-11-13T17:13:12Z)
• Error Correction Code Transformer [92.10654749898927]
  We propose to extend for the first time the Transformer architecture to the soft decoding of linear codes at arbitrary block lengths. We encode each channel's output dimension to high dimension for better representation of the bits information to be processed separately. The proposed approach demonstrates the extreme power and flexibility of Transformers and outperforms existing state-of-the-art neural decoders by large margins at a fraction of their time complexity.
  arXiv  Detail & Related papers  (2022-03-27T15:25:58Z)
• MIONet: Learning multiple-input operators via tensor product [2.5426761219054312]
  We study the operator regression via neural networks for multiple-input operators defined on the product of Banach spaces. Based on our theory and a low-rank approximation, we propose a novel neural operator, MIONet, to learn multiple-input operators.
  arXiv  Detail & Related papers  (2022-02-12T20:37:04Z)
• Dynamic Neural Representational Decoders for High-Resolution Semantic Segmentation [98.05643473345474]
  We propose a novel decoder, termed dynamic neural representational decoder (NRD) As each location on the encoder's output corresponds to a local patch of the semantic labels, in this work, we represent these local patches of labels with compact neural networks. This neural representation enables our decoder to leverage the smoothness prior in the semantic label space, and thus makes our decoder more efficient.
  arXiv  Detail & Related papers  (2021-07-30T04:50:56Z)
• On Sparsifying Encoder Outputs in Sequence-to-Sequence Models [90.58793284654692]
  We take Transformer as the testbed and introduce a layer of gates in-between the encoder and the decoder. The gates are regularized using the expected value of the sparsity-inducing L0penalty. We investigate the effects of this sparsification on two machine translation and two summarization tasks.
  arXiv  Detail & Related papers  (2020-04-24T16:57:52Z)
• Correcting spanning errors with a fractal code [7.6146285961466]
  We propose an efficient decoder for the Fibonacci code'; a two-dimensional classical code that mimics the fractal nature of the cubic code. We perform numerical experiments that show our decoder is robust to one-dimensional, correlated errors.
  arXiv  Detail & Related papers  (2020-02-26T19:00:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.