Input convex neural networks as surrogates in mathematical optimisation

• URL: http://arxiv.org/abs/2608.09707v1
• Date: Mon, 10 Aug 2026 15:11:22 GMT
• Title: Input convex neural networks as surrogates in mathematical optimisation
• Abstract summary: We advocate input convex neural networks (ICNNs) as structurally superior surrogates when the underlying response is approximately convex or concave.<n>Case studies on humanitarian food aid, oil well routing, and wine blending show that ICNN surrogates match FNN accuracy and deliver gains in solve time and scalability.
• Score: 5.127528264113274
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Embedding trained neural networks as surrogates within optimisation problems is an established practice in operations research. The prevailing approach uses feedforward neural networks (FNNs) with ReLU activations, whose piecewise-linear structure admits an exact but computationally intensive mixed-integer programming (MIP) reformulation as the networks grow. We advocate input convex neural networks (ICNNs) as structurally superior surrogates when the underlying response is approximately convex or concave. The convex architecture offers two computational advantages. First, the ICNN-MIP formulation tends to yield a tighter linear programming (LP) relaxation than its FNN-MIP counterpart, with no integrality gap in favourable instances. Second, ICNNs uniquely admit an LP-based reformulation via epigraph representations of ReLU activations, though this embedding is not always exact. When it is not, we exploit the properties of ICNNs to construct the strongest continuous relaxation over box domains, namely, the convex hull of the ICNN's graph, bounded below by the epigraph and above by the concave envelope; this construction is tractable under input convexity but hard for general ReLU networks. On this basis, we develop a branch-and-bound algorithm that builds this relaxation at each node, branches directly on input variables rather than intermediate variables as in MIP reformulations, and terminates at the root node whenever the epigraph embedding is valid. Case studies on humanitarian food aid, oil well routing, and wine blending show that ICNN surrogates match FNN accuracy and deliver gains in solve time and scalability, supporting ICNN as the default surrogate when the underlying function is convex, concave, or well-approximated as such.

Related papers

• Reducing ANN-SNN Conversion Error via Residual Membrane Potential Alignment [52.910770415431244]
  Spiking Neural Networks (SNNs) serve as core architectures for neuromorphic computing.<n>ANN-SNN conversion circumvents issues by reusing well-trained ANN weights for low-latency, energy-efficient inference.<n>Existing conversion schemes suffer from severe accuracy drops at small timesteps, large inference delays and cumulative quantization errors.
  arXiv  Detail & Related papers  (2026-08-14T04:46:03Z)
• Hyper Input Convex Neural Networks for Shape Constrained Learning and Optimal Transport [9.150567695563067]
  We introduce Hyperdimensional Input Convex Neural Networks (HyCNNs), a novel neural network architecture designed for learning convex functions.<n>HyCNNs combine the principles of Maxout networks with input convex neural networks (ICNNs) to create a neural network that is always convex in the input, theoretically capable of leveraging depth, and performs reliable when trained at scale compared to ICNNs.
  arXiv  Detail & Related papers  (2026-04-29T17:52:05Z)
• SOC-ICNN: From Polyhedral to Conic Geometry for Learning Convex Surrogate Functions [13.0197279214062]
  SOC-ICNN is an architecture that generalizes the underlying optimization class from LP to Second-Order Cone Programming (SOCP)<n>By explicitly injecting positive semi-definite curvature and Euclidean norm-based conic primitives, our formulation introduces native curvature smooth representation while preserving a rigorous optimization-theoretic interpretation.<n>Experiments demonstrate that SOC-ICNN substantially improves function approximation, while delivering competitive downstream decision quality.
  arXiv  Detail & Related papers  (2026-04-24T08:43:32Z)
• Efficient Encrypted Computation in Convolutional Spiking Neural Networks with TFHE [1.3047787405425166]
  Homomorphic Encryption (FHE) is a crucial technology for privacy-preserving computation.<n>FHE struggles with continuous non-polynomial functions, as it operates on discrete integers and supports only addition and multiplication.<n>We introduce FHE-DiCSNN, a framework built on the TFHE scheme, utilizing the discrete nature of SNNs for secure and efficient computations.
  arXiv  Detail & Related papers  (2026-03-25T03:15:59Z)
• Convexity in ReLU Neural Networks: beyond ICNNs? [17.01649106055384]
  We show that every convex function implemented by a 1-hidden-layer ReLU network can be expressed by an ICNN with the same architecture.<n>We also provide a numerical procedure that allows an exact check of convexity for ReLU neural networks with a large number of affine regions.
  arXiv  Detail & Related papers  (2025-01-06T13:53:59Z)
• Deep-Unrolling Multidimensional Harmonic Retrieval Algorithms on Neuromorphic Hardware [78.17783007774295]
  This paper explores the potential of conversion-based neuromorphic algorithms for highly accurate and energy-efficient single-snapshot multidimensional harmonic retrieval.<n>A novel method for converting the complex-valued convolutional layers and activations into spiking neural networks (SNNs) is developed.<n>The converted SNNs achieve almost five-fold power efficiency at moderate performance loss compared to the original CNNs.
  arXiv  Detail & Related papers  (2024-12-05T09:41:33Z)
• Neural Network Verification with Branch-and-Bound for General Nonlinearities [63.39918329535165]
  Branch-and-bound (BaB) is among the most effective techniques for neural network (NN) verification.<n>We develop a general framework, named GenBaB, to conduct BaB on general nonlinearities to verify NNs with general architectures.<n>Our framework also allows the verification of general nonlinear graphs and enables verification applications beyond simple NNs.
  arXiv  Detail & Related papers  (2024-05-31T17:51:07Z)
• Fixing the NTK: From Neural Network Linearizations to Exact Convex Programs [63.768739279562105]
  We show that for a particular choice of mask weights that do not depend on the learning targets, this kernel is equivalent to the NTK of the gated ReLU network on the training data. A consequence of this lack of dependence on the targets is that the NTK cannot perform better than the optimal MKL kernel on the training set.
  arXiv  Detail & Related papers  (2023-09-26T17:42:52Z)
• BNN-DP: Robustness Certification of Bayesian Neural Networks via Dynamic Programming [8.162867143465382]
  We introduce BNN-DP, an efficient framework for analysis of adversarial robustness of Bayesian Neural Networks. We show that BNN-DP outperforms state-of-the-art methods by up to four orders of magnitude in both tightness of the bounds and computational efficiency.
  arXiv  Detail & Related papers  (2023-06-19T07:19:15Z)
• Benign Overfitting in Deep Neural Networks under Lazy Training [72.28294823115502]
  We show that when the data distribution is well-separated, DNNs can achieve Bayes-optimal test error for classification. Our results indicate that interpolating with smoother functions leads to better generalization.
  arXiv  Detail & Related papers  (2023-05-30T19:37:44Z)
• Comparative Analysis of Interval Reachability for Robust Implicit and Feedforward Neural Networks [64.23331120621118]
  We use interval reachability analysis to obtain robustness guarantees for implicit neural networks (INNs) INNs are a class of implicit learning models that use implicit equations as layers. We show that our approach performs at least as well as, and generally better than, applying state-of-the-art interval bound propagation methods to INNs.
  arXiv  Detail & Related papers  (2022-04-01T03:31:27Z)
• Modeling from Features: a Mean-field Framework for Over-parameterized Deep Neural Networks [54.27962244835622]
  This paper proposes a new mean-field framework for over- parameterized deep neural networks (DNNs) In this framework, a DNN is represented by probability measures and functions over its features in the continuous limit. We illustrate the framework via the standard DNN and the Residual Network (Res-Net) architectures.
  arXiv  Detail & Related papers  (2020-07-03T01:37:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.