Learning to Solve Related Linear Systems

• URL: http://arxiv.org/abs/2503.17265v1
• Date: Fri, 21 Mar 2025 16:05:45 GMT
• Title: Learning to Solve Related Linear Systems
• Authors: Disha Hegde, Jon Cockayne,
• Abstract summary: We propose a novel probabilistic linear solver over the parameter space.<n>We leverage information from the solved linear systems in a regression setting to provide an efficient posterior mean and covariance.<n>We advocate using this as companion regression model for the preconditioned conjugate gradient method.
• Score: 1.2891210250935148
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Solving multiple parametrised related systems is an essential component of many numerical tasks. Borrowing strength from the solved systems and learning will make this process faster. In this work, we propose a novel probabilistic linear solver over the parameter space. This leverages information from the solved linear systems in a regression setting to provide an efficient posterior mean and covariance. We advocate using this as companion regression model for the preconditioned conjugate gradient method, and discuss the favourable properties of the posterior mean and covariance as the initial guess and preconditioner. We also provide several design choices for this companion solver. Numerical experiments showcase the benefits of using our novel solver in a hyperparameter optimisation problem.

Related papers

• A General Bayesian Framework for Informative Input Design in System Identification [86.05414211113627]
  We tackle the problem of informative input design for system identification.<n>We select inputs, observe the corresponding outputs from the true system, and optimize the parameters of our model to best fit the data.<n>Our method outperforms model-free baselines with various linear and nonlinear dynamics.
  arXiv  Detail & Related papers  (2025-01-28T01:57:51Z)
• Learning to Relax: Setting Solver Parameters Across a Sequence of Linear System Instances [42.16343861129583]
  We show that a bandit online learning algorithm can select parameters for a sequence of instances such that the overall cost approaches that of the best fixed $omega$ as the sequence length increases. Our work provides the first learning-theoretic treatment of high-precision linear system solvers and the first end-to-end guarantees for data-driven scientific computing.
  arXiv  Detail & Related papers  (2023-10-03T17:51:42Z)
• An Optimization-based Deep Equilibrium Model for Hyperspectral Image Deconvolution with Convergence Guarantees [71.57324258813675]
  We propose a novel methodology for addressing the hyperspectral image deconvolution problem. A new optimization problem is formulated, leveraging a learnable regularizer in the form of a neural network. The derived iterative solver is then expressed as a fixed-point calculation problem within the Deep Equilibrium framework.
  arXiv  Detail & Related papers  (2023-06-10T08:25:16Z)
• OKRidge: Scalable Optimal k-Sparse Ridge Regression [21.17964202317435]
  We propose a fast algorithm, OKRidge, for sparse ridge regression. We also propose a method to warm-start our solver, which leverages a beam search.
  arXiv  Detail & Related papers  (2023-04-13T17:34:44Z)
• Message Passing Neural PDE Solvers [60.77761603258397]
  We build a neural message passing solver, replacing allally designed components in the graph with backprop-optimized neural function approximators. We show that neural message passing solvers representationally contain some classical methods, such as finite differences, finite volumes, and WENO schemes. We validate our method on various fluid-like flow problems, demonstrating fast, stable, and accurate performance across different domain topologies, equation parameters, discretizations, etc., in 1D and 2D.
  arXiv  Detail & Related papers  (2022-02-07T17:47:46Z)
• Solving weakly supervised regression problem using low-rank manifold regularization [77.34726150561087]
  We solve a weakly supervised regression problem. Under "weakly" we understand that for some training points the labels are known, for some unknown, and for others uncertain due to the presence of random noise or other reasons such as lack of resources. In the numerical section, we applied the suggested method to artificial and real datasets using Monte-Carlo modeling.
  arXiv  Detail & Related papers  (2021-04-13T23:21:01Z)
• QROSS: QUBO Relaxation Parameter Optimisation via Learning Solver Surrogates [14.905085636501438]
  We build surrogate models of QUBO solvers via learning from solver data on a collection of instances of a problem. In this way, we are able capture the common structure of the instances and their interactions with the solver, and produce good choices of penalty parameters. QROSS is shown to generalise well to out-of-distribution datasets and different types of QUBO solvers.
  arXiv  Detail & Related papers  (2021-03-19T09:06:12Z)
• Divide and Learn: A Divide and Conquer Approach for Predict+Optimize [50.03608569227359]
  The predict+optimize problem combines machine learning ofproblem coefficients with a optimization prob-lem that uses the predicted coefficients. We show how to directlyexpress the loss of the optimization problem in terms of thepredicted coefficients as a piece-wise linear function. We propose a novel divide and algorithm to tackle optimization problems without this restriction and predict itscoefficients using the optimization loss.
  arXiv  Detail & Related papers  (2020-12-04T00:26:56Z)
• Computational characteristics of feedforward neural networks for solving a stiff differential equation [0.0]
  We study the solution of a simple but fundamental stiff ordinary differential equation modelling a damped system. We show that it is possible to identify preferable choices to be made for parameters and methods. Overall we extend the current literature in the field by showing what can be done in order to obtain reliable and accurate results by the neural network approach.
  arXiv  Detail & Related papers  (2020-12-03T12:22:24Z)
• Resampling with neural networks for stochastic parameterization in multiscale systems [0.0]
  We present a machine-learning method, used for the conditional resampling of observations or reference data from a fully resolved simulation. It is based on the probabilistic classiffcation of subsets of reference data, conditioned on macroscopic variables. We validate our approach on the Lorenz 96 system, using two different parameter settings.
  arXiv  Detail & Related papers  (2020-04-03T10:09:18Z)
• Implicit differentiation of Lasso-type models for hyperparameter optimization [82.73138686390514]
  We introduce an efficient implicit differentiation algorithm, without matrix inversion, tailored for Lasso-type problems. Our approach scales to high-dimensional data by leveraging the sparsity of the solutions.
  arXiv  Detail & Related papers  (2020-02-20T18:43:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.