Neural Legendre-Fenchel transform with Hessian Preconditioning

• URL: http://arxiv.org/abs/2606.09077v1
• Date: Mon, 08 Jun 2026 06:21:44 GMT
• Title: Neural Legendre-Fenchel transform with Hessian Preconditioning
• Abstract summary: This work builds on the reformulation of the Legendre-Fenchel transform as a projective polarity.<n>A residual network near the identity can then learn this simplified mapping.<n> Experiments on a diverse set of convex functions, including high-dimensional benchmarks, demonstrate improved convergence rates.
• Score: 6.467090475885798
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The Legendre-Fenchel (LF) transform is a fundamental tool in convex analysis and machine learning that maps lower semi-continuous functions to their convex conjugates. In practice, when closed-form formula are not available for expressing convex conjugates of given functions, one must approximate them using various techniques. One recent such versatile numerical method is the deep Legendre transform method which relies on neural networks although it remains challenging particularly for tackling ill-conditioned functions. This work builds on the reformulation of the LF transform as a projective polarity. A notable property of this framework is its affine invariance. We leverage this affine invariance to introduce a Hessian-based preconditioning strategy. Specifically, we apply an affine deformation around a minimizer so that the second-order Taylor approximation of the function coincides with the canonical paraboloid, whose conjugation map is the identity. A residual network initialized near the identity can then learn this simplified mapping, while the original conjugation map is recovered through the inverse deformation. The proposed preconditioning incurs only a modest computational overhead, consisting of a single eigendecomposition during initialization and two matrix-vector multiplications per query. Experiments on a diverse set of convex functions, including high-dimensional benchmarks, demonstrate improved convergence rates and enhanced numerical accuracy of the conjugation, with particularly significant gains for ill-conditioned problems. Finally, we discuss the scope of applicability of our proposed method and highlight several of its limitations.

Related papers

• Difference-of-Convex Regularization for Graph Learning by Differentiable Programming [1.9651461643699661]
  We propose a Difference-of-Convex Regularizer (DCR) graph learning framework that approximates the spectral action of the Laplacian pseudoinverse without direct inversion.<n>By reformulating Laplacian-Regularized Nonnegative Least Squares (LR-NNLS) through a dual representation, DCR decouples pseudoinverse learning from instance-specific inference.<n>We establish theoretical guarantees on stability and the existence of a unique fixed point for DCR algorithm.
  arXiv  Detail & Related papers  (2026-08-13T03:10:05Z)
• Deep Legendre Transform [0.7734726150561086]
  We introduce a novel deep learning algorithm for computing convex conjugates of differentiable convex functions.<n>Our approach facilitates an efficient gradient-based framework for the minimization of approximation errors.<n> Numerical experiments demonstrate our method's ability to deliver accurate results across different high-dimensional examples.
  arXiv  Detail & Related papers  (2025-12-22T18:22:11Z)
• Preconditioned subgradient method for composite optimization: overparameterization and fast convergence [16.437253140200788]
  Composite optimization problems involve minimizing the composition of a smooth map with a convex function.<n>The subgradient method achieves local linear convergence when the composite loss is well-conditioned.<n>We introduce a Levenberg-Morrison-Marquardt subgradient method that converges linearly under mild regularity conditions.
  arXiv  Detail & Related papers  (2025-09-15T00:26:47Z)
• DiffuMatch: Category-Agnostic Spectral Diffusion Priors for Robust Non-rigid Shape Matching [53.39693288324375]
  We show that both in-network regularization and functional map training can be replaced with data-driven methods.<n>We first train a generative model of functional maps in the spectral domain using score-based generative modeling.<n>We then exploit the resulting model to promote the structural properties of ground truth functional maps on new shape collections.
  arXiv  Detail & Related papers  (2025-07-31T16:44:54Z)
• Generalized Linear Mode Connectivity for Transformers [87.32299363530996]
  A striking phenomenon is linear mode connectivity (LMC), where independently trained models can be connected by low- or zero-loss paths.<n>Prior work has predominantly focused on neuron re-ordering through permutations, but such approaches are limited in scope.<n>We introduce a unified framework that captures four symmetry classes: permutations, semi-permutations, transformations, and general invertible maps.<n>This generalization enables, for the first time, the discovery of low- and zero-barrier linear paths between independently trained Vision Transformers and GPT-2 models.
  arXiv  Detail & Related papers  (2025-06-28T01:46:36Z)
• EulerFormer: Sequential User Behavior Modeling with Complex Vector Attention [88.45459681677369]
  We propose a novel transformer variant with complex vector attention, named EulerFormer. It provides a unified theoretical framework to formulate both semantic difference and positional difference. It is more robust to semantic variations and possesses moresuperior theoretical properties in principle.
  arXiv  Detail & Related papers  (2024-03-26T14:18:43Z)
• Stable Nonconvex-Nonconcave Training via Linear Interpolation [51.668052890249726]
  This paper presents a theoretical analysis of linearahead as a principled method for stabilizing (large-scale) neural network training. We argue that instabilities in the optimization process are often caused by the nonmonotonicity of the loss landscape and show how linear can help by leveraging the theory of nonexpansive operators.
  arXiv  Detail & Related papers  (2023-10-20T12:45:12Z)
• Object Representations as Fixed Points: Training Iterative Refinement Algorithms with Implicit Differentiation [88.14365009076907]
  Iterative refinement is a useful paradigm for representation learning. We develop an implicit differentiation approach that improves the stability and tractability of training.
  arXiv  Detail & Related papers  (2022-07-02T10:00:35Z)
• Orthogonalizing Convolutional Layers with the Cayley Transform [83.73855414030646]
  We propose and evaluate an alternative approach to parameterize convolutional layers that are constrained to be orthogonal. We show that our method indeed preserves orthogonality to a high degree even for large convolutions.
  arXiv  Detail & Related papers  (2021-04-14T23:54:55Z)
• AMITE: A Novel Polynomial Expansion for Analyzing Neural Network Nonlinearities [1.8761314918771685]
  Polynomial expansions are important in the analysis of neural network nonlinearities. Existing approaches span classical Taylor and Chebyshev methods. There are no approaches that provide a consistent method an expansion with all these properties.
  arXiv  Detail & Related papers  (2020-07-13T07:58:47Z)
• Invertible Generative Modeling using Linear Rational Splines [11.510009152620666]
  Normalizing flows attempt to model an arbitrary probability distribution through a set of invertible mappings. The first flow designs used coupling layer mappings built upon affine transformations. Intrepid piecewise functions as a replacement for affine transformations have attracted attention.
  arXiv  Detail & Related papers  (2020-01-15T08:05:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.