Effective Covariance Dynamics in Solvable High-Dimensional GANs

• URL: http://arxiv.org/abs/2606.27246v1
• Date: Thu, 25 Jun 2026 16:33:54 GMT
• Title: Effective Covariance Dynamics in Solvable High-Dimensional GANs
• Abstract summary: We study a solvable high-dimensional model of generative adversarial network (GAN) training in which a linear generator learns a low-dimensional subspace from data.<n>We prove that the microscopic training process converges, in the high-dimensional limit, to deterministic ordinary differential equations governed by this effective covariance.
• Score: 2.639628765667673
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We study a solvable high-dimensional model of generative adversarial network (GAN) training in which a linear generator learns a low-dimensional subspace from data with structured latent covariance. Prior solvable GAN analyses assume unconditional signals with diagonal latent covariance; we extend the multi-feature discriminator setting to class-dependent, correlated, and non-zero-mean latent structure. For the quadratic energy discriminator, all such heterogeneity enters the dynamics through a probability-weighted effective second moment. We prove that the stochastic microscopic training process converges, in the high-dimensional limit, to deterministic ordinary differential equations governed by this effective covariance. In the matched-covariance specialization, the stability analysis yields a mode-wise solvable interval determined by the learning rates and noise level: learning begins when the leading effective eigenvalue crosses the lower threshold, while full recovery requires all relevant effective modes to remain within the interval. This reveals a signal-boosting mechanism: low-rank correlations can lift weak directions above the learnability threshold, whereas overly strong correlations destabilize recovery. Numerical simulations validate the ODE, phase boundary, and boosting mechanism. Experiments on MNIST, FashionMNIST, and CIFAR-10 further show that informed generator covariance improves alignment with the data-driven reference subspace.

Related papers

• Digital Twin-Driven Adaptive Sim-to-Real Alignment via Reinforcement Learning for Vibration-Based Bearing Health Monitoring Under Data Scarcity [7.28808693425028]
  vibration-based health monitoring of rotating machinery requires reliable fault diagnosis under operational data constraints.<n>We formulate feature alignment as a continuous-action Markov decision process solved via Proximal Policy Optimization.<n>An asymmetry-aware strategy reserves real data for the Normal class while augmenting fault classes with policy-aligned simulated samples.
  arXiv  Detail & Related papers  (2026-06-23T08:47:24Z)
• Clipping Bottleneck: Stabilizing RLVR via Stochastic Recovery of Near-Boundary Signals [83.0127582612634]
  Near-boundary Rescue (NSR) is a minimal, plug-and-play modification that retains out-of-bound tokens to recover lost signals.<n>NSR substantially improves training stability and delivers consistent gains over strong baselines such as DAPO and GSPO.
  arXiv  Detail & Related papers  (2026-05-21T16:45:31Z)
• Operator-Guided Invariance Learning for Continuous Reinforcement Learning [8.820825533010543]
  Reinforcement learning (RL) with continuous time and state/action spaces is often data-intensive and brittle under nuisance variability and shift.<n>We propose textbfVPSD-RL (Value-Preserving Structure Discovery for Reinforcement Learning)<n>It models continuous RL as a controlled diffusion with value-preserving mappings defined through Lie-group actions and associated pullback operators.<n>We show that a value-preserving structure exists exactly when pulling back the value function and pushing forward actions commute with the controlled generator and reward functional.
  arXiv  Detail & Related papers  (2026-05-07T16:18:37Z)
• Spectral-Transport Stability and Benign Overfitting in Interpolating Learning [0.0]
  We develop a theoretical framework for generalization in the interpolating regime of statistical learning.<n>We prove finite-sample risk bounds, establish a sharp benign-overfitting index, and derive explicit phase-transition rates under spectral decay.<n>The framework also clarifies implicit regularization by showing how dynamics can select interpolating solutions of minimal spectral-transport energy.
  arXiv  Detail & Related papers  (2026-04-09T15:19:00Z)
• Binary Flow Matching: Prediction-Loss Space Alignment for Robust Learning [23.616336786063552]
  Flow matching has emerged as a powerful framework for generative modeling.<n>We identify a latent structural mismatch that arises when it is coupled with velocity-based objectives.<n>We prove that re-aligning the objective to the signal space eliminates the singular weighting.
  arXiv  Detail & Related papers  (2026-02-11T02:02:30Z)
• Equivariant Evidential Deep Learning for Interatomic Potentials [55.6997213490859]
  Uncertainty quantification is critical for assessing the reliability of machine learning interatomic potentials in molecular dynamics simulations.<n>Existing UQ approaches for MLIPs are often limited by high computational cost or suboptimal performance.<n>We propose textitEquivariant Evidential Deep Learning for Interatomic Potentials ($texte2$IP), a backbone-agnostic framework that models atomic forces and their uncertainty jointly.
  arXiv  Detail & Related papers  (2026-02-11T02:00:25Z)
• On the Joint Minimization of Regularization Loss Functions in Deep Variational Bayesian Methods for Attribute-Controlled Symbolic Music Generation [47.38557855930304]
  Explicit latent variable models provide a flexible yet powerful framework for data synthesis.<n>We show that existing approaches struggle to jointly minimize both regularization objectives.<n>We show that suitable attribute transformations can help achieve both controllability and regularization of the target latent dimensions.
  arXiv  Detail & Related papers  (2025-11-10T14:09:25Z)
• Exact Dynamics of Multi-class Stochastic Gradient Descent [4.1538344141902135]
  We develop a framework for analyzing the training and learning rate dynamics on a variety of high-dimensional optimization problems trained using one-pass gradient descent (SGD)<n>We give exact expressions for a large class of functions of the limiting dynamics, including the risk and the overlap with the true signal, in terms of a deterministic solution to a system of ODEs.
  arXiv  Detail & Related papers  (2025-10-15T20:31:49Z)
• The High Line: Exact Risk and Learning Rate Curves of Stochastic Adaptive Learning Rate Algorithms [8.681909776958184]
  We develop a framework for analyzing the training and learning rate dynamics on a large class of high-dimensional optimization problems. We give exact expressions for the risk and learning rate curves in terms of a deterministic solution to a system of ODEs. We investigate in detail two adaptive learning rates -- an idealized exact line search and AdaGrad-Norm on the least squares problem.
  arXiv  Detail & Related papers  (2024-05-30T00:27:52Z)
• Provable Guarantees for Generative Behavior Cloning: Bridging Low-Level Stability and High-Level Behavior [51.60683890503293]
  We propose a theoretical framework for studying behavior cloning of complex expert demonstrations using generative modeling. We show that pure supervised cloning can generate trajectories matching the per-time step distribution of arbitrary expert trajectories.
  arXiv  Detail & Related papers  (2023-07-27T04:27:26Z)
• Identifiability and Asymptotics in Learning Homogeneous Linear ODE Systems from Discrete Observations [114.17826109037048]
  Ordinary Differential Equations (ODEs) have recently gained a lot of attention in machine learning. theoretical aspects, e.g., identifiability and properties of statistical estimation are still obscure. This paper derives a sufficient condition for the identifiability of homogeneous linear ODE systems from a sequence of equally-spaced error-free observations sampled from a single trajectory.
  arXiv  Detail & Related papers  (2022-10-12T06:46:38Z)
• Training Generative Adversarial Networks by Solving Ordinary Differential Equations [54.23691425062034]
  We study the continuous-time dynamics induced by GAN training. From this perspective, we hypothesise that instabilities in training GANs arise from the integration error. We experimentally verify that well-known ODE solvers (such as Runge-Kutta) can stabilise training.
  arXiv  Detail & Related papers  (2020-10-28T15:23:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.