Breaking Symmetries of the Reservoir Equations in Echo State Networks

• URL: http://arxiv.org/abs/2010.07103v2
• Date: Fri, 4 Dec 2020 10:28:07 GMT
• Title: Breaking Symmetries of the Reservoir Equations in Echo State Networks
• Abstract summary: We find that the simplest popular setup has a harmful symmetry, which leads to the prediction of what we call mirror-attractor. Similar problems can arise in a general context, and we use them to explain the success or failure of some designs.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reservoir computing has repeatedly been shown to be extremely successful in the prediction of nonlinear time-series. However, there is no complete understanding of the proper design of a reservoir yet. We find that the simplest popular setup has a harmful symmetry, which leads to the prediction of what we call mirror-attractor. We prove this analytically. Similar problems can arise in a general context, and we use them to explain the success or failure of some designs. The symmetry is a direct consequence of the hyperbolic tangent activation function. Further, four ways to break the symmetry are compared numerically: A bias in the output, a shift in the input, a quadratic term in the readout, and a mixture of even and odd activation functions. Firstly, we test their susceptibility to the mirror-attractor. Secondly, we evaluate their performance on the task of predicting Lorenz data with the mean shifted to zero. The short-time prediction is measured with the forecast horizon while the largest Lyapunov exponent and the correlation dimension are used to represent the climate. Finally, the same analysis is repeated on a combined dataset of the Lorenz attractor and the Halvorsen attractor, which we designed to reveal potential problems with symmetry. We find that all methods except the output bias are able to fully break the symmetry with input shift and quadratic readout performing the best overall.

Related papers

• Symmetry-initialized quantum Gibbs sampling: a non-Abelian asymmetry cascade [0.0]
  For a quantum Gibbs sampler whose mixing bottleneck is a weakly broken symmetry, I prove a general speedup-versus-factor dichotomy.<n>I then show that when the bottleneck is a weakly broken symmetry, the otherwise exponentially expensive bottleneck eigenvector is the symmetry charge.<n>I verify these results analytically and numerically in an $SU(2)$ Davies sampler.
  arXiv  Detail & Related papers  (2026-07-27T06:03:15Z)
• Entangling Power and Symmetries in the Quantum Rabi Model [0.0]
  We show that the time-averaged entangling power responds to both the hidden symmetry and the $U(1)$ symmetry in the Rabi family.<n>Results demonstrate that the entangling power can serve as an operator diagnostic to reveal the presence and properties of hidden and manifest symmetries in light-matter systems.
  arXiv  Detail & Related papers  (2026-07-13T18:11:55Z)
• Target localization, identification and sensing using latent symmetries [0.0]
  This is the first time latent symmetries have been exploited successfully for sensing problems.<n>It is also the first time latent symmetries have been observed in a three-dimensional open system that cannot be approximated by a sparse graph.
  arXiv  Detail & Related papers  (2026-05-31T19:42:13Z)
• Aligning Network Equivariance with Data Symmetry: A Theoretical Framework and Adaptive Approach for Image Restoration [71.51826063983434]
  Image restoration is an inherently ill posed inverse problem.<n>We use non strict symmetry as a constraint to formulate the restoration inverse problem.<n>We show that the equivariance error of the optimal restoration operator is strictly bounded by the data symmetry error and the discretization mesh size.
  arXiv  Detail & Related papers  (2026-05-13T16:22:19Z)
• Implicit bias as a Gauge correction: Theory and Inverse Design [2.9379512315137117]
  A central problem in machine learning theory is to characterize how learning dynamics select particular solutions compatible with the training objective.<n>We identify a general mechanism, in terms of an explicit correction of the learning dynamics, for the emergence of implicit biases.<n>We compute the resulting induced bias for a range of dynamics, showing how several well known results fit into a single unified framework.
  arXiv  Detail & Related papers  (2026-01-10T15:33:09Z)
• Unregularized Linear Convergence in Zero-Sum Game from Preference Feedback [50.89125374999765]
  We provide the first convergence guarantee for Optimistic Multiplicative Weights Update ($mathtOMWU$) in NLHF.<n>Our analysis identifies a novel marginal convergence behavior, where the probability of rarely played actions grows exponentially from exponentially small values.
  arXiv  Detail & Related papers  (2025-12-31T12:08:29Z)
• Translation symmetry restoration in integrable systems: the noninteracting case [0.16385815610837165]
  We study translation symmetry restoration in integrable systems.<n>In particular, we consider non-interacting spinless fermions on the lattice prepared in non-equilibrium states invariant under $nu>1$ lattice shifts.<n>We show that, differently from random unitary circuits where symmetry restoration occurs abruptly for times proportional to the subsystem size, here symmetry is restored smoothly and over timescales of the order of the subsystem size squared.
  arXiv  Detail & Related papers  (2025-06-17T14:11:31Z)
• Computing Game Symmetries and Equilibria That Respect Them [77.72705755558839]
  We study the computational of identifying and using symmetries in games. We find a strong connection between game symmetries and graph automorphisms. We show that finding a Nash equilibrium that respects a given set of symmetries is exactly as hard as Brouwer fixed point and gradient descent problems.
  arXiv  Detail & Related papers  (2025-01-15T16:15:16Z)
• Strong-to-weak Symmetry Breaking and Entanglement Transitions [6.095133268764501]
  We study the entanglement transition from the perspective of strong-to-weak symmetry breaking. Our results provide a novel viewpoint on the entanglement transition under symmetry constraints.
  arXiv  Detail & Related papers  (2024-11-08T06:42:28Z)
• Learning Infinitesimal Generators of Continuous Symmetries from Data [15.42275880523356]
  We propose a novel symmetry learning algorithm based on transformations defined with one- parameter groups. Our method is built upon minimal inductive biases, encompassing not only commonly utilized symmetries rooted in Lie groups but also extending to symmetries derived from nonlinear generators.
  arXiv  Detail & Related papers  (2024-10-29T08:28:23Z)
• Noether's razor: Learning Conserved Quantities [16.81984465529089]
  We parameterise symmetries as learnable conserved quantities. We then allow conserved quantities and associated symmetries to be learned directly from train data. We find that our method correctly identifies the correct conserved quantities and U($n$) and SE($n$) symmetry groups.
  arXiv  Detail & Related papers  (2024-10-10T16:29:49Z)
• Probing the effects of broken symmetries in machine learning [0.0]
  We show that non-symmetric models can learn symmetries from data, and that doing so can even be beneficial for the accuracy of the model. We focus specifically on physical observables that are likely to be affected -- directly or indirectly -- by symmetry breaking, finding negligible consequences when the model is used in an interpolative, bulk, regime.
  arXiv  Detail & Related papers  (2024-06-25T17:34:09Z)
• A U-turn on Double Descent: Rethinking Parameter Counting in Statistical Learning [68.76846801719095]
  We show that double descent appears exactly when and where it occurs, and that its location is not inherently tied to the threshold p=n. This provides a resolution to tensions between double descent and statistical intuition.
  arXiv  Detail & Related papers  (2023-10-29T12:05:39Z)
• Third quantization of open quantum systems: new dissipative symmetries and connections to phase-space and Keldysh field theory formulations [77.34726150561087]
  We reformulate the technique of third quantization in a way that explicitly connects all three methods. We first show that our formulation reveals a fundamental dissipative symmetry present in all quadratic bosonic or fermionic Lindbladians. For bosons, we then show that the Wigner function and the characteristic function can be thought of as ''wavefunctions'' of the density matrix.
  arXiv  Detail & Related papers  (2023-02-27T18:56:40Z)
• Modeling the space-time correlation of pulsed twin beams [68.8204255655161]
  Entangled twin-beams generated by parametric down-conversion are among the favorite sources for imaging-oriented applications. We propose a semi-analytic model which aims to bridge the gap between time-consuming numerical simulations and the unrealistic plane-wave pump theory.
  arXiv  Detail & Related papers  (2023-01-18T11:29:49Z)
• Implicit Bias of Gradient Descent on Reparametrized Models: On Equivalence to Mirror Descent [64.26008239544085]
  gradient flow with any commuting parametrization is equivalent to continuous mirror descent with a related Legendre function. continuous mirror descent with any Legendre function can be viewed as gradient flow with a related commuting parametrization.
  arXiv  Detail & Related papers  (2022-07-08T17:47:11Z)
• Emergent XY* transition driven by symmetry fractionalization and anyon condensation [0.0]
  We study the phase diagram and anyon condensation transitions of a $mathbbZ$ topological order perturbed by Ising interactions in the Toric Code. The interplay between the global Ising symmetry and the lattice space group symmetries results in a non-trivial symmetry fractionalization class for the anyons. We provide numerical evidence for the occurrence of two symmetry breaking patterns predicted by the specific symmetry fractionalization class of the anyons in the explored phase diagram.
  arXiv  Detail & Related papers  (2022-04-07T18:00:00Z)
• Analytic Signal Phase in $N-D$ by Linear Symmetry Tensor--fingerprint modeling [69.35569554213679]
  We show that the Analytic Signal phase, and its gradient have a hitherto unstudied discontinuity in $2-D $ and higher dimensions. This shortcoming can result in severe artifacts whereas the problem does not exist in $1-D $ signals. We suggest the use of Linear Symmetry phase, relying on more than one set of Gabor filters, but with a negligible computational add-on.
  arXiv  Detail & Related papers  (2020-05-16T21:17:26Z)
• Continuous-time quantum walks in the presence of a quadratic perturbation [55.41644538483948]
  We address the properties of continuous-time quantum walks with Hamiltonians of the form $mathcalH= L + lambda L2$. We consider cycle, complete, and star graphs because paradigmatic models with low/high connectivity and/or symmetry.
  arXiv  Detail & Related papers  (2020-05-13T14:53:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.