Exact Dual Geometry of SOC-ICNN Value Functions

• URL: http://arxiv.org/abs/2605.04722v1
• Date: Wed, 06 May 2026 10:14:36 GMT
• Title: Exact Dual Geometry of SOC-ICNN Value Functions
• Abstract summary: We study the exact first-order and local second-order geometry of SOC-ICNNs from the dual viewpoint.<n>We show that supporting slopes, subdifferentials, directional derivatives, and local Hessians can be recovered directly from optimal dual variables.
• Score: 15.23698901433138
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Input Convex Neural Networks (ICNNs) are commonly used in a two-stage manner: one first trains a convex network and then minimizes it over its input in a downstream inference problem. Recent second-order-cone ICNNs (SOC-ICNNs) enrich ReLU-based ICNNs with quadratic and conic modules and admit an exact representation as value functions of second-order cone programs (SOCPs). This value-function structure enables an explicit convex-analytic treatment of SOC-ICNN inference. In this paper, we study the exact first-order and local second-order geometry of SOC-ICNNs from the dual viewpoint. We show that supporting slopes, subdifferentials, directional derivatives, and local Hessians can be recovered directly from optimal dual variables. These results provide the geometric primitives for white-box SOC-ICNN inference, going beyond black-box automatic differentiation. Numerical experiments validate the exact multiplier readout, the local Hessian formula, and the set-valued behavior at structurally degenerate inputs. We also provide a step-by-step tutorial showing how the readout mechanism instantiates a complete white-box inference loop. The code is available at https://anonymous.4open.science/r/SOC-ICNN-Theory-BEFC/.

Related papers

• Reducing ANN-SNN Conversion Error via Residual Membrane Potential Alignment [52.910770415431244]
  Spiking Neural Networks (SNNs) serve as core architectures for neuromorphic computing.<n>ANN-SNN conversion circumvents issues by reusing well-trained ANN weights for low-latency, energy-efficient inference.<n>Existing conversion schemes suffer from severe accuracy drops at small timesteps, large inference delays and cumulative quantization errors.
  arXiv  Detail & Related papers  (2026-08-14T04:46:03Z)
• Input convex neural networks as surrogates in mathematical optimisation [5.127528264113274]
  We advocate input convex neural networks (ICNNs) as structurally superior surrogates when the underlying response is approximately convex or concave.<n>Case studies on humanitarian food aid, oil well routing, and wine blending show that ICNN surrogates match FNN accuracy and deliver gains in solve time and scalability.
  arXiv  Detail & Related papers  (2026-08-10T15:11:22Z)
• SOC-ICNN: From Polyhedral to Conic Geometry for Learning Convex Surrogate Functions [13.0197279214062]
  SOC-ICNN is an architecture that generalizes the underlying optimization class from LP to Second-Order Cone Programming (SOCP)<n>By explicitly injecting positive semi-definite curvature and Euclidean norm-based conic primitives, our formulation introduces native curvature smooth representation while preserving a rigorous optimization-theoretic interpretation.<n>Experiments demonstrate that SOC-ICNN substantially improves function approximation, while delivering competitive downstream decision quality.
  arXiv  Detail & Related papers  (2026-04-24T08:43:32Z)
• S$^2$NN: Sub-bit Spiking Neural Networks [53.08060832135342]
  Spiking Neural Networks (SNNs) offer an energy-efficient paradigm for machine intelligence.<n>Despite recent advances in binary SNNs, the storage and computational demands remain substantial for large-scale networks.<n>We propose Sub-bit Spiking Neural Networks (S$2$NNs) that represent weights with less than one bit.
  arXiv  Detail & Related papers  (2025-09-29T04:17:44Z)
• Automated discovery of finite volume schemes using Graph Neural Networks [2.867517731896504]
  We establish that Graph Neural Networks (GNNs) can serve purposes beyond their traditional role.<n>We show that a GNN trained on a dataset consisting solely of two-node graphs can extrapolate a first-order Finite Volume scheme.<n>Using symbolic regression, we show that the network effectively rediscovers the exact analytical formulation of the standard first-order FV scheme.
  arXiv  Detail & Related papers  (2025-08-26T14:08:46Z)
• Input Convex Kolmogorov Arnold Networks [0.0]
  This article presents an input convex neural network architecture using Kolmogorov-Arnold networks (ICKAN)<n>Two specific networks are presented: the first is based on a low-order, linear-by-part, representation of functions, and a universal approximation theorem is provided.<n>We demonstrate on simple tests that these networks perform competitively with classical input convex neural networks (ICNNs)
  arXiv  Detail & Related papers  (2025-05-27T13:56:01Z)
• Scalable Graph Compressed Convolutions [68.85227170390864]
  We propose a differentiable method that applies permutations to calibrate input graphs for Euclidean convolution. Based on the graph calibration, we propose the Compressed Convolution Network (CoCN) for hierarchical graph representation learning.
  arXiv  Detail & Related papers  (2024-07-26T03:14:13Z)
• Dual Convexified Convolutional Neural Networks [27.0231994885228]
  We propose the framework of dual convexified convolutional neural networks (DCCNNs) In this framework, we first introduce a primal learning problem motivated from convexified convolutional neural networks (CCNNs) We then construct the dual convex training program through careful analysis of the Karush-Kuhn-Tucker (KKT) conditions and Fenchel conjugates.
  arXiv  Detail & Related papers  (2022-05-27T15:45:08Z)
• Rethinking Nearest Neighbors for Visual Classification [56.00783095670361]
  k-NN is a lazy learning method that aggregates the distance between the test image and top-k neighbors in a training set. We adopt k-NN with pre-trained visual representations produced by either supervised or self-supervised methods in two steps. Via extensive experiments on a wide range of classification tasks, our study reveals the generality and flexibility of k-NN integration.
  arXiv  Detail & Related papers  (2021-12-15T20:15:01Z)
• Connecting Weighted Automata, Tensor Networks and Recurrent Neural Networks through Spectral Learning [58.14930566993063]
  We present connections between three models used in different research fields: weighted finite automata(WFA) from formal languages and linguistics, recurrent neural networks used in machine learning, and tensor networks. We introduce the first provable learning algorithm for linear 2-RNN defined over sequences of continuous vectors input.
  arXiv  Detail & Related papers  (2020-10-19T15:28:00Z)
• Implicit Convex Regularizers of CNN Architectures: Convex Optimization of Two- and Three-Layer Networks in Polynomial Time [70.15611146583068]
  We study training of Convolutional Neural Networks (CNNs) with ReLU activations. We introduce exact convex optimization with a complexity with respect to the number of data samples, the number of neurons, and data dimension.
  arXiv  Detail & Related papers  (2020-06-26T04:47:20Z)
• Deep Networks as Logical Circuits: Generalization and Interpretation [10.223907995092835]
  We present a hierarchical decomposition of the Deep Neural Networks (DNNs) discrete classification map into logical (AND/OR) combinations of intermediate (True/False) classifiers of the input. We show that the learned, internal, logical computations correspond to semantically meaningful categories that allow DNN descriptions in plain English.
  arXiv  Detail & Related papers  (2020-03-25T20:39:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.