An efficient Quasi-Newton method for nonlinear inverse problems via learned singular values

• URL: http://arxiv.org/abs/2012.07676v2
• Date: Mon, 1 Mar 2021 07:58:13 GMT
• Title: An efficient Quasi-Newton method for nonlinear inverse problems via learned singular values
• Abstract summary: We present a highly efficient data-driven Quasi-Newton method applicable to nonlinear inverse problems. We achieve this by using the singular value decomposition and learning a mapping from model outputs to the singular values to compute the updated Jacobian. This enables a speed-up expected of Quasi-Newton methods without accumulating roundoff errors.
• Score: 16.135488140265775
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Solving complex optimization problems in engineering and the physical sciences requires repetitive computation of multi-dimensional function derivatives. Commonly, this requires computationally-demanding numerical differentiation such as perturbation techniques, which ultimately limits the use for time-sensitive applications. In particular, in nonlinear inverse problems Gauss-Newton methods are used that require iterative updates to be computed from the Jacobian. Computationally more efficient alternatives are Quasi-Newton methods, where the repeated computation of the Jacobian is replaced by an approximate update. Here we present a highly efficient data-driven Quasi-Newton method applicable to nonlinear inverse problems. We achieve this, by using the singular value decomposition and learning a mapping from model outputs to the singular values to compute the updated Jacobian. This enables a speed-up expected of Quasi-Newton methods without accumulating roundoff errors, enabling time-critical applications and allowing for flexible incorporation of prior knowledge necessary to solve ill-posed problems. We present results for the highly non-linear inverse problem of electrical impedance tomography with experimental data.

Related papers

• Extraction of informative statistical features in the problem of forecasting time series generated by It{ô}-type processes [65.01025489527173]
  We use the parameters of statistically adjusted mixture-type models of the observed regularities of the behavior of the time series.<n>We show that the use of additional statistical features improves the prediction.
  arXiv  Detail & Related papers  (2026-04-18T06:31:21Z)
• Multi-Dimensional Visual Data Recovery: Scale-Aware Tensor Modeling and Accelerated Randomized Computation [51.65236537605077]
  We propose a new type of network compression optimization technique, fully randomized tensor network compression (FCTN)<n>FCTN has significant advantages in correlation characterization and transpositional in algebra, and has notable achievements in multi-dimensional data processing and analysis.<n>We derive efficient algorithms with guarantees to solve the formulated models.
  arXiv  Detail & Related papers  (2026-02-13T14:56:37Z)
• A Neural-Operator Preconditioned Newton Method for Accelerated Nonlinear Solvers [4.9270397649918]
  We introduce a fixed-point neural operator (FPNO) that learns the direct mapping from the current iterate to the solution by emulating fixed-point iterations.<n>Unlike traditional line-search or trust-region algorithms, the proposed FPNO adaptively employs negative step sizes to effectively mitigate the effects of unbalanced nonlinearities.
  arXiv  Detail & Related papers  (2025-11-11T22:26:38Z)
• Symmetric Rank-One Quasi-Newton Methods for Deep Learning Using Cubic Regularization [0.5120567378386615]
  First-order descent and other first-order variants, such as Adam and AdaGrad, are commonly used in the field of deep learning. However, these methods do not exploit curvature information. Quasi-Newton methods re-use previously computed low Hessian approximations.
  arXiv  Detail & Related papers  (2025-02-17T20:20:11Z)
• Regularized dynamical parametric approximation of stiff evolution problems [0.0]
  We study a class of nonlinear parametrizations $ u(t) = Phi(theta(t)) $, where the evolving parameters $theta(t)$ are to be computed. The primary focus is on tackling the challenges posed by the combination of stiff evolution problems and irregular parametrizations.
  arXiv  Detail & Related papers  (2025-01-21T13:29:36Z)
• Bayesian Model Parameter Learning in Linear Inverse Problems with Application in EEG Focal Source Imaging [49.1574468325115]
  Inverse problems can be described as limited-data problems in which the signal of interest cannot be observed directly. We studied a linear inverse problem that included an unknown non-linear model parameter. We utilized a Bayesian model-based learning approach that allowed signal recovery and subsequently estimation of the model parameter.
  arXiv  Detail & Related papers  (2025-01-07T18:14:24Z)
• A fast neural hybrid Newton solver adapted to implicit methods for nonlinear dynamics [6.642649934130245]
  We propose a novel operator learning based hybrid Newton's method to accelerate this solution of the nonlinear time step system for stiff time-evolution nonlinear equations. A quantifiable rate of improvement in Newton's method is provided and we analyse the upper bound of the generalisation error of our unsupervised learning strategy.
  arXiv  Detail & Related papers  (2024-07-04T14:02:10Z)
• A Structure-Guided Gauss-Newton Method for Shallow ReLU Neural Network [18.06366638807982]
  We propose a structure-guided Gauss-Newton (SgGN) method for solving least squares problems using a shallow ReLU neural network. The method effectively takes advantage of both the least squares structure and the neural network structure of the objective function.
  arXiv  Detail & Related papers  (2024-04-07T20:24:44Z)
• Adaptive pruning-based Newton's method for distributed learning [14.885388389215587]
  This paper presents a novel and efficient algorithm named Distributed Adaptive Newton Learning (textttDANL) textttDANL attains a linear convergence rate while efficiently adapting to available resources and keeping high efficiency. Experiments demonstrate that textttDANL achieves linear convergence with efficient communication and strong performance across different datasets.
  arXiv  Detail & Related papers  (2023-08-20T04:01:30Z)
• An Optimization-based Deep Equilibrium Model for Hyperspectral Image Deconvolution with Convergence Guarantees [71.57324258813675]
  We propose a novel methodology for addressing the hyperspectral image deconvolution problem. A new optimization problem is formulated, leveraging a learnable regularizer in the form of a neural network. The derived iterative solver is then expressed as a fixed-point calculation problem within the Deep Equilibrium framework.
  arXiv  Detail & Related papers  (2023-06-10T08:25:16Z)
• Neural incomplete factorization: learning preconditioners for the conjugate gradient method [2.899792823251184]
  We develop a data-driven approach to accelerate the generation of effective preconditioners. We replace the typically hand-engineered preconditioners by the output of graph neural networks. Our method generates an incomplete factorization of the matrix and is, therefore, referred to as neural incomplete factorization (NeuralIF)
  arXiv  Detail & Related papers  (2023-05-25T11:45:46Z)
• Hybridized Methods for Quantum Simulation in the Interaction Picture [69.02115180674885]
  We provide a framework that allows different simulation methods to be hybridized and thereby improve performance for interaction picture simulations. Physical applications of these hybridized methods yield a gate complexity scaling as $log2 Lambda$ in the electric cutoff. For the general problem of Hamiltonian simulation subject to dynamical constraints, these methods yield a query complexity independent of the penalty parameter $lambda$ used to impose an energy cost.
  arXiv  Detail & Related papers  (2021-09-07T20:01:22Z)
• The Neural Network shifted-Proper Orthogonal Decomposition: a Machine Learning Approach for Non-linear Reduction of Hyperbolic Equations [0.0]
  In this work we approach the problem of automatically detecting the correct pre-processing transformation in a statistical learning framework. The purely data-driven method allowed us to generalise the existing approaches of linear subspace manipulation to non-linear hyperbolic problems with unknown advection fields. The proposed algorithm has been validated against simple test cases to benchmark its performances and later successfully applied to a multiphase simulation.
  arXiv  Detail & Related papers  (2021-08-14T15:13:35Z)
• Recent advances in Bayesian optimization with applications to parameter reconstruction in optical nano-metrology [0.0]
  reconstruction is a common problem in optical nano metrology. We present a Bayesian Target Vector Optimization scheme which combines two approaches. We find that the presented method generally uses fewer calls of the model function than any of the competing schemes to achieve similar reconstruction performance.
  arXiv  Detail & Related papers  (2021-07-12T15:32:15Z)
• DiffPD: Differentiable Projective Dynamics with Contact [65.88720481593118]
  We present DiffPD, an efficient differentiable soft-body simulator with implicit time integration. We evaluate the performance of DiffPD and observe a speedup of 4-19 times compared to the standard Newton's method in various applications.
  arXiv  Detail & Related papers  (2021-01-15T00:13:33Z)
• Quantum-Inspired Algorithms from Randomized Numerical Linear Algebra [53.46106569419296]
  We create classical (non-quantum) dynamic data structures supporting queries for recommender systems and least-squares regression. We argue that the previous quantum-inspired algorithms for these problems are doing leverage or ridge-leverage score sampling in disguise.
  arXiv  Detail & Related papers  (2020-11-09T01:13:07Z)
• Learning Fast Approximations of Sparse Nonlinear Regression [50.00693981886832]
  In this work, we bridge the gap by introducing the Threshold Learned Iterative Shrinkage Algorithming (NLISTA) Experiments on synthetic data corroborate our theoretical results and show our method outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-10-26T11:31:08Z)
• Non-parametric Models for Non-negative Functions [48.7576911714538]
  We provide the first model for non-negative functions from the same good linear models. We prove that it admits a representer theorem and provide an efficient dual formulation for convex problems.
  arXiv  Detail & Related papers  (2020-07-08T07:17:28Z)
• Multipole Graph Neural Operator for Parametric Partial Differential Equations [57.90284928158383]
  One of the main challenges in using deep learning-based methods for simulating physical systems is formulating physics-based data. We propose a novel multi-level graph neural network framework that captures interaction at all ranges with only linear complexity. Experiments confirm our multi-graph network learns discretization-invariant solution operators to PDEs and can be evaluated in linear time.
  arXiv  Detail & Related papers  (2020-06-16T21:56:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.