Neural Fixed-Point Acceleration for Convex Optimization

• URL: http://arxiv.org/abs/2107.10254v2
• Date: Fri, 23 Jul 2021 17:43:00 GMT
• Title: Neural Fixed-Point Acceleration for Convex Optimization
• Authors: Shobha Venkataraman, Brandon Amos
• Abstract summary: We present neural fixed-point acceleration which combines ideas from meta-learning and classical acceleration methods. We apply our framework to SCS, the state-of-the-art solver for convex cone programming.
• Score: 10.06435200305151
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Fixed-point iterations are at the heart of numerical computing and are often a computational bottleneck in real-time applications that typically need a fast solution of moderate accuracy. We present neural fixed-point acceleration which combines ideas from meta-learning and classical acceleration methods to automatically learn to accelerate fixed-point problems that are drawn from a distribution. We apply our framework to SCS, the state-of-the-art solver for convex cone programming, and design models and loss functions to overcome the challenges of learning over unrolled optimization and acceleration instabilities. Our work brings neural acceleration into any optimization problem expressible with CVXPY. The source code behind this paper is available at https://github.com/facebookresearch/neural-scs

Related papers

• Metamizer: a versatile neural optimizer for fast and accurate physics simulations [4.717325308876749]
  We introduce Metamizer, a novel neural network that iteratively solves a wide range of physical systems with high accuracy. We demonstrate that Metamizer achieves unprecedented accuracy for deep learning based approaches. Our results suggest that Metamizer could have a profound impact on future numerical solvers.
  arXiv  Detail & Related papers  (2024-10-10T11:54:31Z)
• Mechanic: A Learning Rate Tuner [52.4242550204696]
  We introduce a technique for tuning the learning rate scale factor of any base optimization algorithm and schedule automatically, which we call textscmechanic. We rigorously evaluate textscmechanic on a range of large scale deep learning tasks with varying batch sizes, schedules, and base optimization algorithms.
  arXiv  Detail & Related papers  (2023-05-31T19:32:43Z)
• NeuralStagger: Accelerating Physics-constrained Neural PDE Solver with Spatial-temporal Decomposition [67.46012350241969]
  This paper proposes a general acceleration methodology called NeuralStagger. It decomposing the original learning tasks into several coarser-resolution subtasks. We demonstrate the successful application of NeuralStagger on 2D and 3D fluid dynamics simulations.
  arXiv  Detail & Related papers  (2023-02-20T19:36:52Z)
• Learning to Optimize Permutation Flow Shop Scheduling via Graph-based Imitation Learning [70.65666982566655]
  Permutation flow shop scheduling (PFSS) is widely used in manufacturing systems. We propose to train the model via expert-driven imitation learning, which accelerates convergence more stably and accurately. Our model's network parameters are reduced to only 37% of theirs, and the solution gap of our model towards the expert solutions decreases from 6.8% to 1.3% on average.
  arXiv  Detail & Related papers  (2022-10-31T09:46:26Z)
• Fast and Robust Non-Rigid Registration Using Accelerated Majorization-Minimization [35.66014845211251]
  Non-rigid registration, which deforms a source shape in a non-rigid way to align with a target shape, is a classical problem in computer vision. Existing methods typically adopt the $ell_p$ type robust norm to measure the alignment error and regularize the smoothness of deformation. We propose a formulation for robust non-rigid registration based on a globally smooth robust norm for alignment and regularization.
  arXiv  Detail & Related papers  (2022-06-07T16:00:33Z)
• Joint inference and input optimization in equilibrium networks [68.63726855991052]
  deep equilibrium model is a class of models that foregoes traditional network depth and instead computes the output of a network by finding the fixed point of a single nonlinear layer. We show that there is a natural synergy between these two settings. We demonstrate this strategy on various tasks such as training generative models while optimizing over latent codes, training models for inverse problems like denoising and inpainting, adversarial training and gradient based meta-learning.
  arXiv  Detail & Related papers  (2021-11-25T19:59:33Z)
• Accelerating GMRES with Deep Learning in Real-Time [0.0]
  We show a real-time machine learning algorithm that can be used to accelerate the time-to-solution for GMRES. Our framework is novel in that is integrates the deep learning algorithm in an in situ fashion.
  arXiv  Detail & Related papers  (2021-03-19T18:21:38Z)
• GradInit: Learning to Initialize Neural Networks for Stable and Efficient Training [59.160154997555956]
  We present GradInit, an automated and architecture method for initializing neural networks. It is based on a simple agnostic; the variance of each network layer is adjusted so that a single step of SGD or Adam results in the smallest possible loss value. It also enables training the original Post-LN Transformer for machine translation without learning rate warmup.
  arXiv  Detail & Related papers  (2021-02-16T11:45:35Z)
• AutoInt: Automatic Integration for Fast Neural Volume Rendering [51.46232518888791]
  We propose a new framework for learning efficient, closed-form solutions to integrals using implicit neural representation networks. We demonstrate a greater than 10x improvement in photorealistic requirements, enabling fast neural volume rendering.
  arXiv  Detail & Related papers  (2020-12-03T05:46:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.