Variational Bounds for Perceptron Learning from Structured Data

• URL: http://arxiv.org/abs/2608.04882v1
• Date: Wed, 05 Aug 2026 14:08:08 GMT
• Title: Variational Bounds for Perceptron Learning from Structured Data
• Abstract summary: We introduce a variational approach to a finite-temperature continuous-spin perceptron trained on a Gaussian mixture.<n>We derive upper minimax variational bounds for the commuted pressure.<n>The same potential yields the fixed-point equations as stationarity.
• Score: 1.8549313085249324
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce a variational approach to a finite-temperature continuous-spin perceptron trained on a Gaussian mixture. The model allows for a broad class of concave utilities and log-concave separable prior measures on the spins. By combining the interpolation method with log-concavity and concentration estimates, we derive lower and upper minimax variational bounds for the limiting quenched pressure. Remarkably, the two bounds differ only in the order of optimization of two variational parameters, while all remaining extrema are controlled by the concave--convex structure of the variational potential. Whenever the two optimizations commute, the two bounds match and identify the solution of the model. The same potential yields the fixed-point equations as stationarity conditions and provides a unified route to the computation of the ground-state energy, training loss, and generalization error.

Related papers

• Residual-Based Time Discretization on Nonlinear Approximation Manifolds: Analysis and Gaussian Applications [0.0]
  We study time-discrete parametric approximations of evolution equations in Hilbert spaces based on residual minimization.<n>Two natural residual formulations are considered: discretization followed by parametrization of the evolution equation, and discretization of the Dirac--Frenkel variational principle governing the parameter dynamics.
  arXiv  Detail & Related papers  (2026-07-16T14:57:44Z)
• Constrained Diffusion Models with Primal-Dual Inference [76.64466170898369]
  We develop constrained diffusion models with primal-dual inference (PDI)<n>PDI infers the optimal primal distribution and its parametrizing dual variable.<n>We evaluate PDI on constrained sampling from a mixture of Gaussians, wireless resource allocation, and portfolio management.
  arXiv  Detail & Related papers  (2026-06-15T18:32:33Z)
• Stability and Generalization of Push-Sum Based Decentralized Optimization over Directed Graphs [55.77845440440496]
  Push-based decentralized communication enables optimization over communication networks, where information exchange may be asymmetric.<n>We develop a unified uniform-stability framework for the Gradient Push (SGP) algorithm.<n>A key technical ingredient is an imbalance-aware generalization bound through two quantities.
  arXiv  Detail & Related papers  (2026-02-24T05:32:03Z)
• Variational Entropic Optimal Transport [67.76725267984578]
  We propose Variational Entropic Optimal Transport (VarEOT) for domain translation problems.<n>VarEOT is based on an exact variational reformulation of the log-partition $log mathbbE[exp(cdot)$ as a tractable generalization over an auxiliary positive normalizer.<n> Experiments on synthetic data and unpaired image-to-image translation demonstrate competitive or improved translation quality.
  arXiv  Detail & Related papers  (2026-02-02T15:48:44Z)
• Analysis of Hessian Scaling for Local and Global Costs in Variational Quantum Algorithm [0.42970700836450487]
  We quantify the entrywise resolvability of the Hessian for Variational Quantum Algorithms.<n>We show two distinct scaling regimes that govern the sample complexity required to resolve Hessian entries against shot noise.
  arXiv  Detail & Related papers  (2026-01-31T15:49:23Z)
• An efficient preconditioned conjugate-gradient solver for a two-component dipolar Bose-Einstein condensate [1.3037017215576174]
  We develop a preconditioned nonlinear conjugate-gradient solver for ground states of binary dipolar Bose-Einstein condensates.<n>We reproduce representative textures and droplet-stability windows reported for dipolar mixtures.
  arXiv  Detail & Related papers  (2025-10-28T15:44:47Z)
• Revisiting Zeroth-Order Optimization: Minimum-Variance Two-Point Estimators and Directionally Aligned Perturbations [57.179679246370114]
  We identify the distribution of random perturbations that minimizes the estimator's variance as the perturbation stepsize tends to zero.<n>Our findings reveal that such desired perturbations can align directionally with the true gradient, instead of maintaining a fixed length.
  arXiv  Detail & Related papers  (2025-10-22T19:06:39Z)
• Neural Optimal Transport Meets Multivariate Conformal Prediction [58.43397908730771]
  We propose a framework for conditional vectorile regression (CVQR)<n>CVQR combines neural optimal transport with quantized optimization, and apply it to predictions.
  arXiv  Detail & Related papers  (2025-09-29T19:50:19Z)
• Variational formulation based on duality to solve partial differential equations: Use of B-splines and machine learning approximants [0.0]
  Many partial differential equations (PDEs) do not have an exact, primal variational structure.<n> variational principle based on the dual (Lagrange multiplier) field was proposed.<n>We derive the dual weak form for the linear, one-dimensional, transient convection-diffusion equation.
  arXiv  Detail & Related papers  (2024-12-02T07:53:47Z)
• Robust scalable initialization for Bayesian variational inference with multi-modal Laplace approximations [0.0]
  Variational mixtures with full-covariance structures suffer from a quadratic growth due to variational parameters with the number of parameters. We propose a method for constructing an initial Gaussian model approximation that can be used to warm-start variational inference.
  arXiv  Detail & Related papers  (2023-07-12T19:30:04Z)
• Neural Control Variates [71.42768823631918]
  We show that a set of neural networks can face the challenge of finding a good approximation of the integrand. We derive a theoretically optimal, variance-minimizing loss function, and propose an alternative, composite loss for stable online training in practice. Specifically, we show that the learned light-field approximation is of sufficient quality for high-order bounces, allowing us to omit the error correction and thereby dramatically reduce the noise at the cost of negligible visible bias.
  arXiv  Detail & Related papers  (2020-06-02T11:17:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.