SOC-ICNN: From Polyhedral to Conic Geometry for Learning Convex Surrogate Functions

• URL: http://arxiv.org/abs/2604.22355v1
• Date: Fri, 24 Apr 2026 08:43:32 GMT
• Title: SOC-ICNN: From Polyhedral to Conic Geometry for Learning Convex Surrogate Functions
• Abstract summary: SOC-ICNN is an architecture that generalizes the underlying optimization class from LP to Second-Order Cone Programming (SOCP)<n>By explicitly injecting positive semi-definite curvature and Euclidean norm-based conic primitives, our formulation introduces native curvature smooth representation while preserving a rigorous optimization-theoretic interpretation.<n>Experiments demonstrate that SOC-ICNN substantially improves function approximation, while delivering competitive downstream decision quality.
• Score: 13.0197279214062
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Classical ReLU-based Input Convex Neural Networks (ICNNs) are equivalent to the optimal value functions of Linear Programming (LP). This intrinsic structural equivalence restricts their representational capacity to piecewise-linear polyhedral functions. To overcome this representational bottleneck, we propose the SOC-ICNN, an architecture that generalizes the underlying optimization class from LP to Second-Order Cone Programming (SOCP). By explicitly injecting positive semi-definite curvature and Euclidean norm-based conic primitives, our formulation introduces native smooth curvature into the representation while preserving a rigorous optimization-theoretic interpretation. We formally prove that SOC-ICNNs strictly expand the representational space of ReLU-ICNNs without increasing the asymptotic order of forward-pass complexity. Extensive experiments demonstrate that SOC-ICNN substantially improves function approximation, while delivering competitive downstream decision quality. The code is available at https://github.com/Kanyooo/SOC-ICNN.

Related papers

• Reducing ANN-SNN Conversion Error via Residual Membrane Potential Alignment [52.910770415431244]
  Spiking Neural Networks (SNNs) serve as core architectures for neuromorphic computing.<n>ANN-SNN conversion circumvents issues by reusing well-trained ANN weights for low-latency, energy-efficient inference.<n>Existing conversion schemes suffer from severe accuracy drops at small timesteps, large inference delays and cumulative quantization errors.
  arXiv  Detail & Related papers  (2026-08-14T04:46:03Z)
• Input convex neural networks as surrogates in mathematical optimisation [5.127528264113274]
  We advocate input convex neural networks (ICNNs) as structurally superior surrogates when the underlying response is approximately convex or concave.<n>Case studies on humanitarian food aid, oil well routing, and wine blending show that ICNN surrogates match FNN accuracy and deliver gains in solve time and scalability.
  arXiv  Detail & Related papers  (2026-08-10T15:11:22Z)
• Exact Dual Geometry of SOC-ICNN Value Functions [15.23698901433138]
  We study the exact first-order and local second-order geometry of SOC-ICNNs from the dual viewpoint.<n>We show that supporting slopes, subdifferentials, directional derivatives, and local Hessians can be recovered directly from optimal dual variables.
  arXiv  Detail & Related papers  (2026-05-06T10:14:36Z)
• Hyper Input Convex Neural Networks for Shape Constrained Learning and Optimal Transport [9.150567695563067]
  We introduce Hyperdimensional Input Convex Neural Networks (HyCNNs), a novel neural network architecture designed for learning convex functions.<n>HyCNNs combine the principles of Maxout networks with input convex neural networks (ICNNs) to create a neural network that is always convex in the input, theoretically capable of leveraging depth, and performs reliable when trained at scale compared to ICNNs.
  arXiv  Detail & Related papers  (2026-04-29T17:52:05Z)
• Barycentric Neural Networks and Length-Weighted Persistent Entropy Loss: A Green Geometric and Topological Framework for Function Approximation [0.0]
  We introduce the Barycentric Neural Network (BNN), a compact shallow architecture that encodes both structure and parameters through a fixed set of base points and their associated barycentric coordinates.<n>We show that the BNN enables the exact representation of continuous piecewise linear functions (CPLFs), ensuring strict continuity across segments.<n>Our approach integrates the BNN with a loss function based on LWPE to optimize the base points that define the BNN, rather than its internal parameters.
  arXiv  Detail & Related papers  (2025-09-08T13:47:21Z)
• Curvature Learning for Generalization of Hyperbolic Neural Networks [51.888534247573894]
  Hyperbolic neural networks (HNNs) have demonstrated notable efficacy in representing real-world data with hierarchical structures.<n>Inappropriate curvatures may cause HNNs to converge to suboptimal parameters, degrading overall performance.<n>We propose a sharpness-aware curvature learning method to smooth the loss landscape, thereby improving the generalization of HNNs.
  arXiv  Detail & Related papers  (2025-08-24T07:14:30Z)
• Convexity in ReLU Neural Networks: beyond ICNNs? [17.01649106055384]
  We show that every convex function implemented by a 1-hidden-layer ReLU network can be expressed by an ICNN with the same architecture.<n>We also provide a numerical procedure that allows an exact check of convexity for ReLU neural networks with a large number of affine regions.
  arXiv  Detail & Related papers  (2025-01-06T13:53:59Z)
• Enhancing GNNs Performance on Combinatorial Optimization by Recurrent Feature Update [0.09986418756990156]
  We introduce a novel algorithm, denoted hereafter as QRF-GNN, leveraging the power of GNNs to efficiently solve Combinatorial optimization (CO) problems. It relies on unsupervised learning by minimizing the loss function derived from QUBO relaxation. Results of experiments show that QRF-GNN drastically surpasses existing learning-based approaches and is comparable to the state-of-the-art conventionals.
  arXiv  Detail & Related papers  (2024-07-23T13:34:35Z)
• The Convex Landscape of Neural Networks: Characterizing Global Optima and Stationary Points via Lasso Models [75.33431791218302]
  Deep Neural Network Network (DNN) models are used for programming purposes. In this paper we examine the use of convex neural recovery models. We show that all the stationary non-dimensional objective objective can be characterized as the standard a global subsampled convex solvers program. We also show that all the stationary non-dimensional objective objective can be characterized as the standard a global subsampled convex solvers program.
  arXiv  Detail & Related papers  (2023-12-19T23:04:56Z)
• Stable Nonconvex-Nonconcave Training via Linear Interpolation [51.668052890249726]
  This paper presents a theoretical analysis of linearahead as a principled method for stabilizing (large-scale) neural network training. We argue that instabilities in the optimization process are often caused by the nonmonotonicity of the loss landscape and show how linear can help by leveraging the theory of nonexpansive operators.
  arXiv  Detail & Related papers  (2023-10-20T12:45:12Z)
• Benign Overfitting in Deep Neural Networks under Lazy Training [72.28294823115502]
  We show that when the data distribution is well-separated, DNNs can achieve Bayes-optimal test error for classification. Our results indicate that interpolating with smoother functions leads to better generalization.
  arXiv  Detail & Related papers  (2023-05-30T19:37:44Z)
• Dual Convexified Convolutional Neural Networks [27.0231994885228]
  We propose the framework of dual convexified convolutional neural networks (DCCNNs) In this framework, we first introduce a primal learning problem motivated from convexified convolutional neural networks (CCNNs) We then construct the dual convex training program through careful analysis of the Karush-Kuhn-Tucker (KKT) conditions and Fenchel conjugates.
  arXiv  Detail & Related papers  (2022-05-27T15:45:08Z)
• Comparative Analysis of Interval Reachability for Robust Implicit and Feedforward Neural Networks [64.23331120621118]
  We use interval reachability analysis to obtain robustness guarantees for implicit neural networks (INNs) INNs are a class of implicit learning models that use implicit equations as layers. We show that our approach performs at least as well as, and generally better than, applying state-of-the-art interval bound propagation methods to INNs.
  arXiv  Detail & Related papers  (2022-04-01T03:31:27Z)
• CDiNN -Convex Difference Neural Networks [0.8122270502556374]
  Neural networks with ReLU activation function have been shown to be universal function approximators learn function mapping as non-smooth functions. New neural network architecture called ICNNs learn the output as a convex input.
  arXiv  Detail & Related papers  (2021-03-31T17:31:16Z)
• Learning to Solve the AC-OPF using Sensitivity-Informed Deep Neural Networks [52.32646357164739]
  We propose a deep neural network (DNN) to solve the solutions of the optimal power flow (ACOPF) The proposed SIDNN is compatible with a broad range of OPF schemes. It can be seamlessly integrated in other learning-to-OPF schemes.
  arXiv  Detail & Related papers  (2021-03-27T00:45:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.