Generalized Inversion of Nonlinear Operators

• URL: http://arxiv.org/abs/2111.10755v3
• Date: Tue, 19 Sep 2023 07:25:51 GMT
• Title: Generalized Inversion of Nonlinear Operators
• Authors: Eyal Gofer and Guy Gilboa
• Abstract summary: Inversion of operators is a fundamental concept in data processing. Most notable is the Moore-Penrose inverse, widely used in physics, statistics, and various fields of engineering.
• Score: 6.191418251390628
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Inversion of operators is a fundamental concept in data processing. Inversion of linear operators is well studied, supported by established theory. When an inverse either does not exist or is not unique, generalized inverses are used. Most notable is the Moore-Penrose inverse, widely used in physics, statistics, and various fields of engineering. This work investigates generalized inversion of nonlinear operators. We first address broadly the desired properties of generalized inverses, guided by the Moore-Penrose axioms. We define the notion for general sets, and then a refinement, termed pseudo-inverse, for normed spaces. We present conditions for existence and uniqueness of a pseudo-inverse and establish theoretical results investigating its properties, such as continuity, its value for operator compositions and projection operators, and others. Analytic expressions are given for the pseudo-inverse of some well-known, non-invertible, nonlinear operators, such as hard- or soft-thresholding and ReLU. We analyze a neural layer and discuss relations to wavelet thresholding. Next, the Drazin inverse, and a relaxation, are investigated for operators with equal domain and range. We present scenarios where inversion is expressible as a linear combination of forward applications of the operator. Such scenarios arise for classes of nonlinear operators with vanishing polynomials, similar to the minimal or characteristic polynomials for matrices. Inversion using forward applications may facilitate the development of new efficient algorithms for approximating generalized inversion of complex nonlinear operators.

Related papers

• Quantum implementation of non-unitary operations with biorthogonal representations [0.0]
  We propose a new dilation method based on the biorthogonal representation of the non-unitary operator. The proposed method excels in implementing non-unitary operators whose eigenvalues have absolute values exceeding one.
  arXiv  Detail & Related papers  (2024-10-29T19:55:02Z)
• Refined Risk Bounds for Unbounded Losses via Transductive Priors [58.967816314671296]
  We revisit the sequential variants of linear regression with the squared loss, classification problems with hinge loss, and logistic regression. Our key tools are based on the exponential weights algorithm with carefully chosen transductive priors.
  arXiv  Detail & Related papers  (2024-10-29T00:01:04Z)
• Optimal Matrix-Mimetic Tensor Algebras via Variable Projection [0.0]
  Matrix mimeticity arises from interpreting tensors as operators that can be multiplied, factorized, and analyzed analogous to matrices. We learn optimal linear mappings and corresponding tensor representations without relying on prior knowledge of the data. We provide original theory of uniqueness of the transformation and convergence analysis of our variable-projection-based algorithm.
  arXiv  Detail & Related papers  (2024-06-11T04:52:23Z)
• General expansion of natural power of linear combination of Bosonic operators in normal order [3.42658286826597]
  We present a general expansion of the natural power of a linear combination of bosonic operators in normal order. Our results have important applications in the study of many-body systems in quantum mechanics.
  arXiv  Detail & Related papers  (2023-05-29T14:26:45Z)
• Marginalized Operators for Off-policy Reinforcement Learning [53.37381513736073]
  Marginalized operators strictly generalize generic multi-step operators, such as Retrace, as special cases. We show that the estimates for marginalized operators can be computed in a scalable way, which also generalizes prior results on marginalized importance sampling as special cases.
  arXiv  Detail & Related papers  (2022-03-30T09:59:59Z)
• Generalized Optimization: A First Step Towards Category Theoretic Learning Theory [1.52292571922932]
  We generalize several optimization algorithms, including a straightforward generalization of gradient descent and a novel generalization of Newton's method. We show that the transformation invariances of these algorithms are preserved. We also show that we can express the change in loss of generalized descent with an inner product-like expression.
  arXiv  Detail & Related papers  (2021-09-20T15:19:06Z)
• Convolutional Filtering and Neural Networks with Non Commutative Algebras [153.20329791008095]
  We study the generalization of non commutative convolutional neural networks. We show that non commutative convolutional architectures can be stable to deformations on the space of operators.
  arXiv  Detail & Related papers  (2021-08-23T04:22:58Z)
• Analytic gradients in variational quantum algorithms: Algebraic extensions of the parameter-shift rule to general unitary transformations [0.0]
  We propose several extensions of the parametric-shift-rule to formulating gradients as linear combinations of expectation values for generators with general eigen-spectrum. Our approaches are exact and do not use any auxiliary qubits, instead they rely on a generator eigen-spectrum analysis.
  arXiv  Detail & Related papers  (2021-07-16T21:37:31Z)
• Robust Differentiable SVD [117.35644933471401]
  Eigendecomposition of symmetric matrices is at the heart of many computer vision algorithms. Instability arises in the presence of eigenvalues that are close to each other. We show that the Taylor expansion of the SVD gradient is theoretically equivalent to the gradient obtained using PI without relying on an iterative process.
  arXiv  Detail & Related papers  (2021-04-08T15:04:15Z)
• A Differential Geometry Perspective on Orthogonal Recurrent Models [56.09491978954866]
  We employ tools and insights from differential geometry to offer a novel perspective on orthogonal RNNs. We show that orthogonal RNNs may be viewed as optimizing in the space of divergence-free vector fields. Motivated by this observation, we study a new recurrent model, which spans the entire space of vector fields.
  arXiv  Detail & Related papers  (2021-02-18T19:39:22Z)
• Supervised Quantile Normalization for Low-rank Matrix Approximation [50.445371939523305]
  We learn the parameters of quantile normalization operators that can operate row-wise on the values of $X$ and/or of its factorization $UV$ to improve the quality of the low-rank representation of $X$ itself. We demonstrate the applicability of these techniques on synthetic and genomics datasets.
  arXiv  Detail & Related papers  (2020-02-08T21:06:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.