Flexible Differentiable Optimization via Model Transformations

• URL: http://arxiv.org/abs/2206.06135v3
• Date: Mon, 31 Jul 2023 08:04:07 GMT
• Title: Flexible Differentiable Optimization via Model Transformations
• Authors: Mathieu Besan\c{c}on and Joaquim Dias Garcia and Beno\^it Legat and Akshay Sharma
• Abstract summary: We introduce DiffOpt, a Julia library to differentiate through the solution of optimization problems with respect to arbitrary parameters present in the objective and/or constraints.
• Score: 1.081463830315253
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce DiffOpt.jl, a Julia library to differentiate through the solution of optimization problems with respect to arbitrary parameters present in the objective and/or constraints. The library builds upon MathOptInterface, thus leveraging the rich ecosystem of solvers and composing well with modeling languages like JuMP. DiffOpt offers both forward and reverse differentiation modes, enabling multiple use cases from hyperparameter optimization to backpropagation and sensitivity analysis, bridging constrained optimization with end-to-end differentiable programming. DiffOpt is built on two known rules for differentiating quadratic programming and conic programming standard forms. However, thanks ability to differentiate through model transformation, the user is not limited to these forms and can differentiate with respect to the parameters of any model that can be reformulated into these standard forms. This notably includes programs mixing affine conic constraints and convex quadratic constraints or objective function.

Related papers

• Optimized QUBO formulation methods for quantum computing [0.4999814847776097]
  We show how to apply our techniques in case of an NP-hard optimization problem inspired by a real-world financial scenario. We follow by submitting instances of this problem to two D-wave quantum annealers, comparing the performances of our novel approach with the standard methods used in these scenarios.
  arXiv  Detail & Related papers  (2024-06-11T19:59:05Z)
• Variational Quantum Framework for Partial Differential Equation Constrained Optimization [0.6138671548064355]
  We present a novel variational quantum framework for PDE constrained optimization problems. The proposed framework utilizes the variational quantum linear (VQLS) algorithm and a black box as its main building blocks.
  arXiv  Detail & Related papers  (2024-05-26T18:06:43Z)
• End-to-End Learning for Fair Multiobjective Optimization Under Uncertainty [55.04219793298687]
  The Predict-Then-Forecast (PtO) paradigm in machine learning aims to maximize downstream decision quality. This paper extends the PtO methodology to optimization problems with nondifferentiable Ordered Weighted Averaging (OWA) objectives. It shows how optimization of OWA functions can be effectively integrated with parametric prediction for fair and robust optimization under uncertainty.
  arXiv  Detail & Related papers  (2024-02-12T16:33:35Z)
• Predict-Then-Optimize by Proxy: Learning Joint Models of Prediction and Optimization [59.386153202037086]
  Predict-Then- framework uses machine learning models to predict unknown parameters of an optimization problem from features before solving. This approach can be inefficient and requires handcrafted, problem-specific rules for backpropagation through the optimization step. This paper proposes an alternative method, in which optimal solutions are learned directly from the observable features by predictive models.
  arXiv  Detail & Related papers  (2023-11-22T01:32:06Z)
• Discrete quadratic model QUBO solution landscapes [0.0]
  We investigate the effects of choice of encoding and penalty strength on the structure of QUBO DQM solution landscapes. This research focuses specifically on one-hot and domain-wall encodings.
  arXiv  Detail & Related papers  (2023-04-30T20:19:46Z)
• Backpropagation of Unrolled Solvers with Folded Optimization [55.04219793298687]
  The integration of constrained optimization models as components in deep networks has led to promising advances on many specialized learning tasks. One typical strategy is algorithm unrolling, which relies on automatic differentiation through the operations of an iterative solver. This paper provides theoretical insights into the backward pass of unrolled optimization, leading to a system for generating efficiently solvable analytical models of backpropagation.
  arXiv  Detail & Related papers  (2023-01-28T01:50:42Z)
• Differentiable Spline Approximations [48.10988598845873]
  Differentiable programming has significantly enhanced the scope of machine learning. Standard differentiable programming methods (such as autodiff) typically require that the machine learning models be differentiable. We show that leveraging this redesigned Jacobian in the form of a differentiable "layer" in predictive models leads to improved performance in diverse applications.
  arXiv  Detail & Related papers  (2021-10-04T16:04:46Z)
• Efficient and Modular Implicit Differentiation [68.74748174316989]
  We propose a unified, efficient and modular approach for implicit differentiation of optimization problems. We show that seemingly simple principles allow to recover many recently proposed implicit differentiation methods and create new ones easily.
  arXiv  Detail & Related papers  (2021-05-31T17:45:58Z)
• Extracting Optimal Solution Manifolds using Constrained Neural Optimization [6.800113407368289]
  Constrained Optimization solution algorithms are restricted to point based solutions. We present an approach for extracting optimal sets as approximate, where unmodified non-informed constraints are defined.
  arXiv  Detail & Related papers  (2020-09-13T15:37:44Z)
• On the Encoder-Decoder Incompatibility in Variational Text Modeling and Beyond [82.18770740564642]
  Variational autoencoders (VAEs) combine latent variables with amortized variational inference. We observe the encoder-decoder incompatibility that leads to poor parameterizations of the data manifold. We propose Coupled-VAE, which couples a VAE model with a deterministic autoencoder with the same structure.
  arXiv  Detail & Related papers  (2020-04-20T10:34:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.