Learning and Concentration for High Dimensional Linear Gaussians: an Invariant Subspace Approach

• URL: http://arxiv.org/abs/2304.01708v1
• Date: Tue, 4 Apr 2023 11:11:26 GMT
• Title: Learning and Concentration for High Dimensional Linear Gaussians: an Invariant Subspace Approach
• Authors: Muhammad Abdullah Naeem
• Abstract summary: We study non-asymptotic bounds on correlation between two time realizations of stable linear systems with isotropic Gaussian noise. Our analysis provide first interpretable and geometric explanation into intricacies of learning and concentration for random dynamical systems.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this work, we study non-asymptotic bounds on correlation between two time realizations of stable linear systems with isotropic Gaussian noise. Consequently, via sampling from a sub-trajectory and using \emph{Talagrands'} inequality, we show that empirical averages of reward concentrate around steady state (dynamical system mixes to when closed loop system is stable under linear feedback policy ) reward , with high-probability. As opposed to common belief of larger the spectral radius stronger the correlation between samples, \emph{large discrepancy between algebraic and geometric multiplicity of system eigenvalues leads to large invariant subspaces related to system-transition matrix}; once the system enters the large invariant subspace it will travel away from origin for a while before coming close to a unit ball centered at origin where an isotropic Gaussian noise can with high probability allow it to escape the current invariant subspace it resides in, leading to \emph{bottlenecks} between different invariant subspaces that span $\mathbb{R}^{n}$, to be precise : system initiated in a large invariant subspace will be stuck there for a long-time: log-linear in dimension of the invariant subspace and inversely to log of inverse of magnitude of the eigenvalue. In the problem of Ordinary Least Squares estimate of system transition matrix via a single trajectory, this phenomenon is even more evident if spectrum of transition matrix associated to large invariant subspace is explosive and small invariant subspaces correspond to stable eigenvalues. Our analysis provide first interpretable and geometric explanation into intricacies of learning and concentration for random dynamical systems on continuous, high dimensional state space; exposing us to surprises in high dimensions

Related papers

• Inferring Kernel $ε$-Machines: Discovering Structure in Complex Systems [49.1574468325115]
  We introduce causal diffusion components that encode the kernel causal-state estimates as a set of coordinates in a reduced dimension space. We show how each component extracts predictive features from data and demonstrate their application on four examples.
  arXiv  Detail & Related papers  (2024-10-01T21:14:06Z)
• Exact dynamics of quantum dissipative $XX$ models: Wannier-Stark localization in the fragmented operator space [49.1574468325115]
  We find an exceptional point at a critical dissipation strength that separates oscillating and non-oscillating decay. We also describe a different type of dissipation that leads to a single decay mode in the whole operator subspace.
  arXiv  Detail & Related papers  (2024-05-27T16:11:39Z)
• On the entanglement of co-ordinate and momentum degrees of freedom in noncommutative space [0.0]
  We investigate the quantum entanglement induced by phase-space noncommutativity. The entanglement properties of coordinate and momentum degrees of freedom are studied. We show that the mere inclusion of non-commutativity of phase-space is not sufficient to generate the entanglement.
  arXiv  Detail & Related papers  (2024-01-05T18:43:47Z)
• From Spectral Theorem to Statistical Independence with Application to System Identification [11.98319841778396]
  We provide first quantitative handle on decay rate of finite powers of state transition matrix $|Ak|$. It is shown that when a stable dynamical system has only one distinct eigenvalue and discrepancy of $n-1$: $|A|$ has a dependence on $n$, resulting dynamics are inseparable. We show that element-wise error is essentially a variant of well-know Littlewood-Offord problem.
  arXiv  Detail & Related papers  (2023-10-16T15:40:43Z)
• Machine learning in and out of equilibrium [58.88325379746631]
  Our study uses a Fokker-Planck approach, adapted from statistical physics, to explore these parallels. We focus in particular on the stationary state of the system in the long-time limit, which in conventional SGD is out of equilibrium. We propose a new variation of Langevin dynamics (SGLD) that harnesses without replacement minibatching.
  arXiv  Detail & Related papers  (2023-06-06T09:12:49Z)
• Continuous percolation in a Hilbert space for a large system of qubits [58.720142291102135]
  The percolation transition is defined through the appearance of the infinite cluster. We show that the exponentially increasing dimensionality of the Hilbert space makes its covering by finite-size hyperspheres inefficient. Our approach to the percolation transition in compact metric spaces may prove useful for its rigorous treatment in other contexts.
  arXiv  Detail & Related papers  (2022-10-15T13:53:21Z)
• Unified Fourier-based Kernel and Nonlinearity Design for Equivariant Networks on Homogeneous Spaces [52.424621227687894]
  We introduce a unified framework for group equivariant networks on homogeneous spaces. We take advantage of the sparsity of Fourier coefficients of the lifted feature fields. We show that other methods treating features as the Fourier coefficients in the stabilizer subgroup are special cases of our activation.
  arXiv  Detail & Related papers  (2022-06-16T17:59:01Z)
• Benign Overfitting of Constant-Stepsize SGD for Linear Regression [122.70478935214128]
  inductive biases are central in preventing overfitting empirically. This work considers this issue in arguably the most basic setting: constant-stepsize SGD for linear regression. We reflect on a number of notable differences between the algorithmic regularization afforded by (unregularized) SGD in comparison to ordinary least squares.
  arXiv  Detail & Related papers  (2021-03-23T17:15:53Z)
• Improved rates for prediction and identification of partially observed linear dynamical systems [4.68299658663016]
  Identification of a linear time-in dynamical system from partial observations is a fundamental problem in control theory. We propose an algorithm that learns such systems with non-asymptotic statistical rates depending on the inherentity $d$ of the system. Our algorithm is based on multi-scale low-rank approximation SVD applied to Hankel matrices of increasing sizes.
  arXiv  Detail & Related papers  (2020-11-19T18:04:18Z)
• Nonlinear entanglement growth in inhomogeneous spacetimes [0.0]
  Entment has become central for the characterization of quantum matter both in and out of equilibrium. We study entanglement dynamics both for the case of noninteracting fermions, allowing for exact numerical solutions, and for random unitary circuits representing a paradigmatic class of ergodic systems.
  arXiv  Detail & Related papers  (2020-06-01T08:58:26Z)
• Polynomially filtered exact diagonalization approach to many-body localization [0.0]
  Polynomially filtered exact diagonalization method (POLFED) for large sparse matrices is introduced. The potential of POLFED is demonstrated examining many-body scaling transition in 1D interacting quantum spin-1/2 chains.
  arXiv  Detail & Related papers  (2020-05-19T15:49:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.