StarNet: Gradient-free Training of Deep Generative Models using Determined System of Linear Equations

• URL: http://arxiv.org/abs/2101.00574v1
• Date: Sun, 3 Jan 2021 08:06:42 GMT
• Title: StarNet: Gradient-free Training of Deep Generative Models using Determined System of Linear Equations
• Authors: Amir Zadeh, Santiago Benoit, Louis-Philippe Morency
• Abstract summary: We present an approach for training deep generative models based on solving determined systems of linear equations. A network that uses this approach, called a StarNet, has the following desirable properties.
• Score: 47.72653430712088
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper we present an approach for training deep generative models solely based on solving determined systems of linear equations. A network that uses this approach, called a StarNet, has the following desirable properties: 1) training requires no gradient as solution to the system of linear equations is not stochastic, 2) is highly scalable when solving the system of linear equations w.r.t the latent codes, and similarly for the parameters of the model, and 3) it gives desirable least-square bounds for the estimation of latent codes and network parameters within each layer.

Related papers

• Two-Point Deterministic Equivalence for Stochastic Gradient Dynamics in Linear Models [76.52307406752556]
  We derive a novel deterministic equivalence for the two-point function of a random resolvent. We give a unified derivation of the performance of a wide variety of high-dimensional trained linear models with gradient descent.
  arXiv  Detail & Related papers  (2025-02-07T16:45:40Z)
• A General Bayesian Framework for Informative Input Design in System Identification [86.05414211113627]
  We tackle the problem of informative input design for system identification. We select inputs, observe the corresponding outputs from the true system, and optimize the parameters of our model to best fit the data. Our method outperforms model-free baselines with various linear and nonlinear dynamics.
  arXiv  Detail & Related papers  (2025-01-28T01:57:51Z)
• Solving Nonlinear Energy Supply and Demand System Using Physics-Informed Neural Networks [1.5728609542259502]
  We propose a method utilizing Physics-Informed Neural Networks (PINNs) to solve the nonlinear energy supply-demand system. We design a neural network with four outputs, where each output approximates a function that corresponds to one of the unknown functions in the nonlinear system of differential equations. The solutions obtained from the neural network for this problem are equivalent when we compare and evaluate them against the Runge-Kutta numerical method of order 4/5 (RK45)
  arXiv  Detail & Related papers  (2024-12-22T12:37:59Z)
• Reconstruction of dynamic systems using genetic algorithms with dynamic search limits [0.0]
  evolutionary computing techniques are presented to estimate the governing equations of a dynamical system using time-series data. Some of the main contributions of the present study are an adequate modification of the genetic algorithm to remove terms with minimal contributions, and a mechanism to escape local optima. Our results demonstrate a reconstruction with an Integral Square Error below 0.22 and a coefficient of determination R-squared of 0.99 for all systems.
  arXiv  Detail & Related papers  (2024-12-03T22:58:25Z)
• Generalized Quadratic Embeddings for Nonlinear Dynamics using Deep Learning [11.339982217541822]
  We present a data-driven methodology for modeling the dynamics of nonlinear systems. In this work, we propose using quadratic systems as the common structure, inspired by the lifting principle.
  arXiv  Detail & Related papers  (2022-11-01T10:03:34Z)
• Unsupervised Reservoir Computing for Solving Ordinary Differential Equations [1.6371837018687636]
  unsupervised reservoir computing (RC), an echo-state recurrent neural network capable of discovering approximate solutions that satisfy ordinary differential equations (ODEs) We use Bayesian optimization to efficiently discover optimal sets in a high-dimensional hyper- parameter space and numerically show that one set is robust and can be used to solve an ODE for different initial conditions and time ranges.
  arXiv  Detail & Related papers  (2021-08-25T18:16:42Z)
• Linear embedding of nonlinear dynamical systems and prospects for efficient quantum algorithms [74.17312533172291]
  We describe a method for mapping any finite nonlinear dynamical system to an infinite linear dynamical system (embedding) We then explore an approach for approximating the resulting infinite linear system with finite linear systems (truncation)
  arXiv  Detail & Related papers  (2020-12-12T00:01:10Z)
• Deep neural network for solving differential equations motivated by Legendre-Galerkin approximation [16.64525769134209]
  We explore the performance and accuracy of various neural architectures on both linear and nonlinear differential equations. We implement a novel Legendre-Galerkin Deep Neural Network (LGNet) algorithm to predict solutions to differential equations.
  arXiv  Detail & Related papers  (2020-10-24T20:25:09Z)
• Multipole Graph Neural Operator for Parametric Partial Differential Equations [57.90284928158383]
  One of the main challenges in using deep learning-based methods for simulating physical systems is formulating physics-based data. We propose a novel multi-level graph neural network framework that captures interaction at all ranges with only linear complexity. Experiments confirm our multi-graph network learns discretization-invariant solution operators to PDEs and can be evaluated in linear time.
  arXiv  Detail & Related papers  (2020-06-16T21:56:22Z)
• Eigendecomposition-Free Training of Deep Networks for Linear Least-Square Problems [107.3868459697569]
  We introduce an eigendecomposition-free approach to training a deep network. We show that our approach is much more robust than explicit differentiation of the eigendecomposition. Our method has better convergence properties and yields state-of-the-art results.
  arXiv  Detail & Related papers  (2020-04-15T04:29:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.