Implementing Recycling Methods for Linear Systems in Python with an Application to Multiple Objective Optimization

• URL: http://arxiv.org/abs/2402.15941v1
• Date: Sun, 25 Feb 2024 00:15:30 GMT
• Title: Implementing Recycling Methods for Linear Systems in Python with an Application to Multiple Objective Optimization
• Authors: Ainara Garcia, Sihong Xie, Arielle Carr
• Abstract summary: We seek to reduce the overall cost of computation when solving linear systems using common recycling methods. In this work, we assessed the performance of recycling minimum residual (RMINRES) method along with a map between coefficient matrices.
• Score: 6.131436685168423
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Sequences of linear systems arise in the predictor-corrector method when computing the Pareto front for multi-objective optimization. Rather than discarding information generated when solving one system, it may be advantageous to recycle information for subsequent systems. To accomplish this, we seek to reduce the overall cost of computation when solving linear systems using common recycling methods. In this work, we assessed the performance of recycling minimum residual (RMINRES) method along with a map between coefficient matrices. For these methods to be fully integrated into the software used in Enouen et al. (2022), there must be working version of each in both Python and PyTorch. Herein, we discuss the challenges we encountered and solutions undertaken (and some ongoing) when computing efficient Python implementations of these recycling strategies. The goal of this project was to implement RMINRES in Python and PyTorch and add it to the established Pareto front code to reduce computational cost. Additionally, we wanted to implement the sparse approximate maps code in Python and PyTorch, so that it can be parallelized in future work.

Related papers

• Sketch 'n Solve: An Efficient Python Package for Large-Scale Least Squares Using Randomized Numerical Linear Algebra [0.0]
  We present Sketch 'n Solve, an open-source Python package that implements efficient randomized numerical linear algebra techniques. We show that our implementation achieves up to 50x speedup over traditional LSQR while maintaining high accuracy, even for ill-conditioned matrices. The package shows particular promise for applications in machine learning optimization, signal processing, and scientific computing.
  arXiv  Detail & Related papers  (2024-09-22T04:29:51Z)
• An Efficient Algorithm for Clustered Multi-Task Compressive Sensing [60.70532293880842]
  Clustered multi-task compressive sensing is a hierarchical model that solves multiple compressive sensing tasks. The existing inference algorithm for this model is computationally expensive and does not scale well in high dimensions. We propose a new algorithm that substantially accelerates model inference by avoiding the need to explicitly compute these covariance matrices.
  arXiv  Detail & Related papers  (2023-09-30T15:57:14Z)
• SurvLIMEpy: A Python package implementing SurvLIME [1.0689187493307983]
  We present SurvLIMEpy, an open-source Python package that implements the SurvLIME algorithm. The package supports a wide variety of survival models, from the Cox Proportional Hazards Model to deep learning models such as DeepHit or DeepSurv.
  arXiv  Detail & Related papers  (2023-02-21T09:54:32Z)
• PyEPO: A PyTorch-based End-to-End Predict-then-Optimize Library for Linear and Integer Programming [9.764407462807588]
  We present the PyEPO package, a PyTorchbased end-to-end predict-then-optimize library in Python. PyEPO is the first such generic tool for linear and integer programming with predicted objective function coefficients.
  arXiv  Detail & Related papers  (2022-06-28T18:33:55Z)
• Fast Differentiable Matrix Square Root and Inverse Square Root [65.67315418971688]
  We propose two more efficient variants to compute the differentiable matrix square root and the inverse square root. For the forward propagation, one method is to use Matrix Taylor Polynomial (MTP), and the other method is to use Matrix Pad'e Approximants (MPA) A series of numerical tests show that both methods yield considerable speed-up compared with the SVD or the NS iteration.
  arXiv  Detail & Related papers  (2022-01-29T10:00:35Z)
• tvopt: A Python Framework for Time-Varying Optimization [1.218340575383456]
  tvopt is a Python framework for prototyping and benchmarking time-varying (or online) optimization algorithms. tvopt provides functionalities for defining both centralized and distributed online problems. The framework implements prediction strategies to improve the accuracy of the online solvers.
  arXiv  Detail & Related papers  (2020-11-12T16:14:09Z)
• PyTorch Distributed: Experiences on Accelerating Data Parallel Training [11.393654219774444]
  PyTorch is a widely-adopted scientific computing package used in deep learning research and applications. This paper presents the design, implementation, and evaluation of the PyTorch distributed data parallel module.
  arXiv  Detail & Related papers  (2020-06-28T20:39:45Z)
• Picasso: A Sparse Learning Library for High Dimensional Data Analysis in R and Python [77.33905890197269]
  We describe a new library which implements a unified pathwise coordinate optimization for a variety of sparse learning problems. The library is coded in R++ and has user-friendly sparse experiments.
  arXiv  Detail & Related papers  (2020-06-27T02:39:24Z)
• Accelerating Feedforward Computation via Parallel Nonlinear Equation Solving [106.63673243937492]
  Feedforward computation, such as evaluating a neural network or sampling from an autoregressive model, is ubiquitous in machine learning. We frame the task of feedforward computation as solving a system of nonlinear equations. We then propose to find the solution using a Jacobi or Gauss-Seidel fixed-point method, as well as hybrid methods of both. Our method is guaranteed to give exactly the same values as the original feedforward computation with a reduced (or equal) number of parallelizable iterations, and hence reduced time given sufficient parallel computing power.
  arXiv  Detail & Related papers  (2020-02-10T10:11:31Z)
• OPFython: A Python-Inspired Optimum-Path Forest Classifier [68.8204255655161]
  This paper proposes a Python-based Optimum-Path Forest framework, denoted as OPFython. As OPFython is a Python-based library, it provides a more friendly environment and a faster prototyping workspace than the C language.
  arXiv  Detail & Related papers  (2020-01-28T15:46:19Z)
• Multi-layer Optimizations for End-to-End Data Analytics [71.05611866288196]
  We introduce Iterative Functional Aggregate Queries (IFAQ), a framework that realizes an alternative approach. IFAQ treats the feature extraction query and the learning task as one program given in the IFAQ's domain-specific language. We show that a Scala implementation of IFAQ can outperform mlpack, Scikit, and specialization by several orders of magnitude for linear regression and regression tree models over several relational datasets.
  arXiv  Detail & Related papers  (2020-01-10T16:14:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.