Basis-to-Basis Operator Learning Using Function Encoders

• URL: http://arxiv.org/abs/2410.00171v1
• Date: Mon, 30 Sep 2024 19:18:34 GMT
• Title: Basis-to-Basis Operator Learning Using Function Encoders
• Authors: Tyler Ingebrand, Adam J. Thorpe, Somdatta Goswami, Krishna Kumar, Ufuk Topcu,
• Abstract summary: We present Basis-to-Basis (B2B) operator learning, a novel approach for learning operators on Hilbert spaces of functions. We derive operator learning algorithms that are directly analogous to eigen-decomposition and singular value decomposition.
• Score: 16.128154294012543
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present Basis-to-Basis (B2B) operator learning, a novel approach for learning operators on Hilbert spaces of functions based on the foundational ideas of function encoders. We decompose the task of learning operators into two parts: learning sets of basis functions for both the input and output spaces, and learning a potentially nonlinear mapping between the coefficients of the basis functions. B2B operator learning circumvents many challenges of prior works, such as requiring data to be at fixed locations, by leveraging classic techniques such as least-squares to compute the coefficients. It is especially potent for linear operators, where we compute a mapping between bases as a single matrix transformation with a closed form solution. Furthermore, with minimal modifications and using the deep theoretical connections between function encoders and functional analysis, we derive operator learning algorithms that are directly analogous to eigen-decomposition and singular value decomposition. We empirically validate B2B operator learning on six benchmark operator learning tasks, and show that it demonstrates a two-orders-of-magnitude improvement in accuracy over existing approaches on several benchmark tasks.

Related papers

• Learning Partial Differential Equations with Deep Parallel Neural Operators [11.121415128908566]
  A novel methodology is to learn an operator as a means of approximating the mapping between outputs. In practical physical science problems, the numerical solutions of partial differential equations are complex. We propose a deep parallel operator model (DPNO) for efficiently and accurately solving partial differential equations.
  arXiv  Detail & Related papers  (2024-09-30T06:04:04Z)
• Operator Learning Using Random Features: A Tool for Scientific Computing [3.745868534225104]
  Supervised operator learning centers on the use of training data to estimate maps between infinite-dimensional spaces. This paper introduces the function-valued random features method. It leads to a supervised operator learning architecture that is practical for nonlinear problems.
  arXiv  Detail & Related papers  (2024-08-12T23:10:39Z)
• Data Complexity Estimates for Operator Learning [4.056627267544063]
  We develop theory to study the data complexity of operator learning. We show that on a narrower class of operators, efficiently approximated by FNO in terms of the number of tunable parameters, efficient operator learning is attainable in data complexity as well.
  arXiv  Detail & Related papers  (2024-05-25T00:16:21Z)
• Neural Operators with Localized Integral and Differential Kernels [77.76991758980003]
  We present a principled approach to operator learning that can capture local features under two frameworks. We prove that we obtain differential operators under an appropriate scaling of the kernel values of CNNs. To obtain local integral operators, we utilize suitable basis representations for the kernels based on discrete-continuous convolutions.
  arXiv  Detail & Related papers  (2024-02-26T18:59:31Z)
• Efficient Model-Free Exploration in Low-Rank MDPs [76.87340323826945]
  Low-Rank Markov Decision Processes offer a simple, yet expressive framework for RL with function approximation. Existing algorithms are either (1) computationally intractable, or (2) reliant upon restrictive statistical assumptions. We propose the first provably sample-efficient algorithm for exploration in Low-Rank MDPs.
  arXiv  Detail & Related papers  (2023-07-08T15:41:48Z)
• Basis Function Encoding of Numerical Features in Factorization Machines for Improved Accuracy [2.3022070933226217]
  We provide a systematic and theoretically-justified way to incorporate numerical features into FM variants. We show that our technique yields a model that learns segmentized functions of the numerical feature spanned by the set of functions of one's choice. Our technique preserves fast training and inference, and requires only a small modification of the computational graph of an FM model.
  arXiv  Detail & Related papers  (2023-05-23T21:10:17Z)
• A Recursively Recurrent Neural Network (R2N2) Architecture for Learning Iterative Algorithms [64.3064050603721]
  We generalize Runge-Kutta neural network to a recurrent neural network (R2N2) superstructure for the design of customized iterative algorithms. We demonstrate that regular training of the weight parameters inside the proposed superstructure on input/output data of various computational problem classes yields similar iterations to Krylov solvers for linear equation systems, Newton-Krylov solvers for nonlinear equation systems, and Runge-Kutta solvers for ordinary differential equations.
  arXiv  Detail & Related papers  (2022-11-22T16:30:33Z)
• Stabilizing Q-learning with Linear Architectures for Provably Efficient Learning [53.17258888552998]
  This work proposes an exploration variant of the basic $Q$-learning protocol with linear function approximation. We show that the performance of the algorithm degrades very gracefully under a novel and more permissive notion of approximation error.
  arXiv  Detail & Related papers  (2022-06-01T23:26:51Z)
• Deep learning with transfer functions: new applications in system identification [0.0]
  This paper presents a linear dynamical operator endowed with a well-defined and efficient back-propagation behavior for automatic derivatives computation. The operator enables end-to-end training of structured networks containing linear transfer functions and other differentiable units.
  arXiv  Detail & Related papers  (2021-04-20T08:58:55Z)
• BCFNet: A Balanced Collaborative Filtering Network with Attention Mechanism [106.43103176833371]
  Collaborative Filtering (CF) based recommendation methods have been widely studied. We propose a novel recommendation model named Balanced Collaborative Filtering Network (BCFNet) In addition, an attention mechanism is designed to better capture the hidden information within implicit feedback and strengthen the learning ability of the neural network.
  arXiv  Detail & Related papers  (2021-03-10T14:59:23Z)
• Multi-task Supervised Learning via Cross-learning [102.64082402388192]
  We consider a problem known as multi-task learning, consisting of fitting a set of regression functions intended for solving different tasks. In our novel formulation, we couple the parameters of these functions, so that they learn in their task specific domains while staying close to each other. This facilitates cross-fertilization in which data collected across different domains help improving the learning performance at each other task.
  arXiv  Detail & Related papers  (2020-10-24T21:35:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.