Flexible Differentiable Optimization via Model Transformations

• URL: http://arxiv.org/abs/2206.06135v3
• Date: Mon, 31 Jul 2023 08:04:07 GMT
• Title: Flexible Differentiable Optimization via Model Transformations
• Abstract summary: We introduce DiffOpt, a Julia library to differentiate through the solution of optimization problems with respect to arbitrary parameters present in the objective and/or constraints.
• Score: 1.081463830315253
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce DiffOpt.jl, a Julia library to differentiate through the solution of optimization problems with respect to arbitrary parameters present in the objective and/or constraints. The library builds upon MathOptInterface, thus leveraging the rich ecosystem of solvers and composing well with modeling languages like JuMP. DiffOpt offers both forward and reverse differentiation modes, enabling multiple use cases from hyperparameter optimization to backpropagation and sensitivity analysis, bridging constrained optimization with end-to-end differentiable programming. DiffOpt is built on two known rules for differentiating quadratic programming and conic programming standard forms. However, thanks ability to differentiate through model transformation, the user is not limited to these forms and can differentiate with respect to the parameters of any model that can be reformulated into these standard forms. This notably includes programs mixing affine conic constraints and convex quadratic constraints or objective function.

Related papers

• Binary Optimization with Complex Constraints via Quantum Approximate Multi-Objective Optimization [1.3827807729367585]
  We show that a class of binary optimization problems with complex non-quadratic objectives or constraints can be reformulated as multi-objective quadratic unconstrained binary optimization problems.<n>We demonstrate the approach for binary portfolio optimization under a Conditional Value-at-Risk constraint.
  arXiv  Detail & Related papers  (2026-09-12T08:07:15Z)
• An End-to-End PyTorch Interface for Differentiable PDE Solvers: A RANS Model-Correction Study [0.0]
  This work presents an end-to-end strategy for solving inverse problems constrained by Partial Differential Equations.<n>We show how to optimize phi within a fully differentiable Python workflow by reformulating the PDE as an implicit layer.<n>A dataset is generated starting from the VKI LS-59 turbine blade geometry using the differentiable BROADCAST solver with the Spalart-Allmaras turbulence model.
  arXiv  Detail & Related papers  (2026-05-19T07:38:50Z)
• Surrogate-Based Differentiable Pipeline for Shape Optimization [64.24199762940444]
  We propose replacing non-differentiable pipeline components with surrogate models which are inherently differentiable.<n>We demonstrate an end-to-end differentiable pipeline where a 3D U-Net full-field surrogate replaces both meshing and simulation steps by training it on the mapping between the signed distance field (SDF) of the shape and the fields of interest.
  arXiv  Detail & Related papers  (2025-11-13T19:30:50Z)
• A General and Streamlined Differentiable Optimization Framework [10.851559133306196]
  This paper presents the DiffOptl interface for Julia optimization framework.<n>A first-class JuMP-native API allows users to obtain named parameters derivatives directly with respect to them.<n>Results demonstrate that differentiing a routine can be as a training tool for experimentation, learning, and design-without deviating from standard JuMP modeling practices.
  arXiv  Detail & Related papers  (2025-10-29T21:42:36Z)
• A Graphical Global Optimization Framework for Parameter Estimation of Statistical Models with Nonconvex Regularization Functions [0.0]
  Problems with linear norm-bound constraints arise in a variety of applications, including portfolio optimization, machine learning, feature selection.<n>We propose a novel graphbased method to globally solve these problems.
  arXiv  Detail & Related papers  (2025-05-06T18:09:54Z)
• $\ell_0$-Regularized Quadratic Surface Support Vector Machines [0.0]
  Kernel-free quadratic surface support vector machines have recently gained traction due to their flexibility in modeling nonlinear decision boundaries without relying on kernel functions.<n>We propose a sparse variant of the QSVM by enforcing a cardinality constraint on the model parameters.<n>We validate our approach on several real-world datasets, demonstrating its ability to reduce overfitting while maintaining strong classification performance.
  arXiv  Detail & Related papers  (2025-01-20T04:26:34Z)
• FERERO: A Flexible Framework for Preference-Guided Multi-Objective Learning [41.95837632934815]
  We introduce a Flexible framEwork for pREfeRence-guided multi-Objective learning (FERERO) To solve this problem, convergent algorithms are developed with both single-loop and primal variants. Experiments on multiple benchmarks demonstrate the proposed method is very competitive in finding preference-guided optimal solutions.
  arXiv  Detail & Related papers  (2024-12-02T18:21:16Z)
• Optimized QUBO formulation methods for quantum computing [0.4999814847776097]
  We show how to apply our techniques in case of an NP-hard optimization problem inspired by a real-world financial scenario. We follow by submitting instances of this problem to two D-wave quantum annealers, comparing the performances of our novel approach with the standard methods used in these scenarios.
  arXiv  Detail & Related papers  (2024-06-11T19:59:05Z)
• Variational Quantum Framework for Partial Differential Equation Constrained Optimization [0.6138671548064355]
  We present a novel variational quantum framework for PDE constrained optimization problems. The proposed framework utilizes the variational quantum linear (VQLS) algorithm and a black box as its main building blocks.
  arXiv  Detail & Related papers  (2024-05-26T18:06:43Z)
• End-to-End Learning for Fair Multiobjective Optimization Under Uncertainty [55.04219793298687]
  The Predict-Then-Forecast (PtO) paradigm in machine learning aims to maximize downstream decision quality. This paper extends the PtO methodology to optimization problems with nondifferentiable Ordered Weighted Averaging (OWA) objectives. It shows how optimization of OWA functions can be effectively integrated with parametric prediction for fair and robust optimization under uncertainty.
  arXiv  Detail & Related papers  (2024-02-12T16:33:35Z)
• Predict-Then-Optimize by Proxy: Learning Joint Models of Prediction and Optimization [59.386153202037086]
  Predict-Then- framework uses machine learning models to predict unknown parameters of an optimization problem from features before solving. This approach can be inefficient and requires handcrafted, problem-specific rules for backpropagation through the optimization step. This paper proposes an alternative method, in which optimal solutions are learned directly from the observable features by predictive models.
  arXiv  Detail & Related papers  (2023-11-22T01:32:06Z)
• Discrete quadratic model QUBO solution landscapes [0.0]
  We investigate the effects of choice of encoding and penalty strength on the structure of QUBO DQM solution landscapes. This research focuses specifically on one-hot and domain-wall encodings.
  arXiv  Detail & Related papers  (2023-04-30T20:19:46Z)
• Backpropagation of Unrolled Solvers with Folded Optimization [55.04219793298687]
  The integration of constrained optimization models as components in deep networks has led to promising advances on many specialized learning tasks. One typical strategy is algorithm unrolling, which relies on automatic differentiation through the operations of an iterative solver. This paper provides theoretical insights into the backward pass of unrolled optimization, leading to a system for generating efficiently solvable analytical models of backpropagation.
  arXiv  Detail & Related papers  (2023-01-28T01:50:42Z)
• Differentiable Spline Approximations [48.10988598845873]
  Differentiable programming has significantly enhanced the scope of machine learning. Standard differentiable programming methods (such as autodiff) typically require that the machine learning models be differentiable. We show that leveraging this redesigned Jacobian in the form of a differentiable "layer" in predictive models leads to improved performance in diverse applications.
  arXiv  Detail & Related papers  (2021-10-04T16:04:46Z)
• Efficient and Modular Implicit Differentiation [68.74748174316989]
  We propose a unified, efficient and modular approach for implicit differentiation of optimization problems. We show that seemingly simple principles allow to recover many recently proposed implicit differentiation methods and create new ones easily.
  arXiv  Detail & Related papers  (2021-05-31T17:45:58Z)
• Extracting Optimal Solution Manifolds using Constrained Neural Optimization [6.800113407368289]
  Constrained Optimization solution algorithms are restricted to point based solutions. We present an approach for extracting optimal sets as approximate, where unmodified non-informed constraints are defined.
  arXiv  Detail & Related papers  (2020-09-13T15:37:44Z)
• On the Encoder-Decoder Incompatibility in Variational Text Modeling and Beyond [82.18770740564642]
  Variational autoencoders (VAEs) combine latent variables with amortized variational inference. We observe the encoder-decoder incompatibility that leads to poor parameterizations of the data manifold. We propose Coupled-VAE, which couples a VAE model with a deterministic autoencoder with the same structure.
  arXiv  Detail & Related papers  (2020-04-20T10:34:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.