Frequency Domain Reservoir Computing

• URL: http://arxiv.org/abs/2606.24969v1
• Date: Tue, 23 Jun 2026 10:14:46 GMT
• Title: Frequency Domain Reservoir Computing
• Abstract summary: Echo State Networks (ESNs) offer a compelling approach by utilizing fixed recurrent weights to circumvent backpropagation through time.<n>We introduce Frequency Domain Reservoir Computing (FRESCO), an ESN architecture operating entirely in the frequency domain.<n> FRESCO drastically reduces computational costs and energy consumption of training and inference.
• Score: 10.316233181797408
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While the quadratic sequence-length bottleneck of transformers has fueled a resurgence in recurrent models, effectively capturing complex dynamics requires architectures that balance efficient training with highly expressive latent states. Echo State Networks (ESNs) offer a compelling approach by utilizing fixed recurrent weights to circumvent backpropagation through time, enabling a closed-form training solution. However, achieving the expressivity needed for complex tasks demands large reservoirs, exposing an $\mathcal{O}(N^2)$ state-update bottleneck that prevents ESNs from matching the scale of contemporary recurrent models. To address this limitation, we introduce Frequency Domain Reservoir Computing (FRESCO), an ESN architecture operating entirely in the frequency domain while avoiding domain-shift overheads to achieve $\mathcal{O}(N)$ complexity for dense, non-linear recurrent updates. By employing a novel dimensional zero-padding input embedding, a packed \FDh readout, and a natively applied frequency-domain non-linearity, FRESCO drastically reduces computational costs and energy consumption of training and inference. Furthermore, FRESCO matches the state-of-the-art predictive performance on memory benchmarks, sequential classification, and multivariate long-horizon forecasting, offering a scalable path forward for dense recurrent architectures.

Related papers

• CRFCAN: A Complex-Valued Cross-Domain Residual Network for Joint Channel and Phase Noise Estimation in Sub-THz OFDM Systems [5.464031544932083]
  In sub-terahertz (sub-THz) communications, the coupling of ultra-wide bandwidth and severe phase noise (PN) impairments renders joint channel and PN estimation highly complex and computationally prohibitive.<n>We propose CRFCAN, a complex-valued residual FFT convolutional attention network designed for joint channel and PN estimation.
  arXiv  Detail & Related papers  (2026-09-10T22:02:08Z)
• FRACTAL: SSM with Fractional Recurrent Architecture for Computational Temporal Analysis of Long Sequences [13.320173396455226]
  This paper proposes a Fractional Recurrent Architecture for Computational Temporal Analysis of Long sequences (FRACTAL)<n>By deriving projection operators with analytically characterized spectral properties and a tunable index, the proposed method sensitivity to recent signal perturbations while preserving the spectral structure that encodes fractional scale-invariant memory dynamics.<n>FRACTAL achieves an average score of 87.11% on the Long Range Arena benchmark, including 61.85% on the ListOps task, outperforming the S5 model.
  arXiv  Detail & Related papers  (2026-05-09T09:38:44Z)
• When Learning Hurts: Fixed-Pole RNN for Real-Time Online Training [58.25341036646294]
  We analytically examine why learning recurrent poles does not provide tangible benefits in data and empirically offer real-time learning scenarios.<n>We show that fixed-pole networks achieve superior performance with lower training complexity, making them more suitable for online real-time tasks.
  arXiv  Detail & Related papers  (2026-02-25T00:15:13Z)
• Distributed physics-informed neural networks via domain decomposition for fast flow reconstruction [8.614942690565782]
  PINs offer a powerful paradigm for flow reconstruction, seamlessly integrating velocity measurements with the governing Navier-N equations to recover complete velocity and latent pressure fields.<n>A critical challenge in such distributed PINs is pressure indeterminacy, where independent sub-networks drift into inconsistent local pressure baselines.<n>By enforcing a unidirectional flow from designated master ranks, our approach eliminates uniqueness and guarantees global pressure while preserving temporal continuity.
  arXiv  Detail & Related papers  (2026-02-05T16:41:55Z)
• Multivariate time series prediction using clustered echo state network [4.4778341776682735]
  Echo state networks (ESNs) offer an efficient alternative to conventional recurrent neural networks.<n>We show that CESNs consistently outperform conventional ESNs in terms of predictive accuracy and robustness to noise.<n>Our algorithm works well across diverse real-world datasets such as the stock market, solar wind, and chaotic Rssler system.
  arXiv  Detail & Related papers  (2025-11-28T17:14:15Z)
• Echo Flow Networks [4.298381633106637]
  We introduce Echo Flow Networks (EFNs), a framework composed of a group of Echo State Networks (X-ESNs) with nonlinear readouts.<n>EFNs achieve up to 4x faster training and 3x smaller model size compared to leading methods like PatchTST.<n>One instantiation of our framework, EchoFormer, consistently achieves new state-of-the-art performance across five benchmark datasets.
  arXiv  Detail & Related papers  (2025-09-28T23:38:55Z)
• ResFormer: All-Time Reservoir Memory for Long Sequence Classification [4.298381633106637]
  Sequence classification is essential in NLP for understanding and categorizing language patterns in tasks like sentiment analysis, intent detection, and topic classification.<n> Transformer-based models, despite achieving state-of-the-art performance, have inherent limitations due to quadratic time and memory complexity.<n>We propose ResFormer, a novel neural network architecture designed to model varying context lengths efficiently through a cascaded methodology.
  arXiv  Detail & Related papers  (2025-09-28T21:20:49Z)
• MesaNet: Sequence Modeling by Locally Optimal Test-Time Training [67.45211108321203]
  We introduce a numerically stable, chunkwise parallelizable version of the recently proposed Mesa layer.<n>We show that optimal test-time training enables reaching lower language modeling perplexity and higher downstream benchmark performance than previous RNNs.
  arXiv  Detail & Related papers  (2025-06-05T16:50:23Z)
• Fast Training of Recurrent Neural Networks with Stationary State Feedbacks [48.22082789438538]
  Recurrent neural networks (RNNs) have recently demonstrated strong performance and faster inference than Transformers.<n>We propose a novel method that replaces BPTT with a fixed gradient feedback mechanism.
  arXiv  Detail & Related papers  (2025-03-29T14:45:52Z)
• Spatial Annealing for Efficient Few-shot Neural Rendering [73.49548565633123]
  We introduce an accurate and efficient few-shot neural rendering method named textbfSpatial textbfAnnealing regularized textbfNeRF (textbfSANeRF)<n>By adding merely one line of code, SANeRF delivers superior rendering quality and much faster reconstruction speed compared to current few-shot neural rendering methods.
  arXiv  Detail & Related papers  (2024-06-12T02:48:52Z)
• TCCT-Net: Two-Stream Network Architecture for Fast and Efficient Engagement Estimation via Behavioral Feature Signals [58.865901821451295]
  We present a novel two-stream feature fusion "Tensor-Convolution and Convolution-Transformer Network" (TCCT-Net) architecture. To better learn the meaningful patterns in the temporal-spatial domain, we design a "CT" stream that integrates a hybrid convolutional-transformer. In parallel, to efficiently extract rich patterns from the temporal-frequency domain, we introduce a "TC" stream that uses Continuous Wavelet Transform (CWT) to represent information in a 2D tensor form.
  arXiv  Detail & Related papers  (2024-04-15T06:01:48Z)
• MF-NeRF: Memory Efficient NeRF with Mixed-Feature Hash Table [62.164549651134465]
  We propose MF-NeRF, a memory-efficient NeRF framework that employs a Mixed-Feature hash table to improve memory efficiency and reduce training time while maintaining reconstruction quality. Our experiments with state-of-the-art Instant-NGP, TensoRF, and DVGO, indicate our MF-NeRF could achieve the fastest training time on the same GPU hardware with similar or even higher reconstruction quality.
  arXiv  Detail & Related papers  (2023-04-25T05:44:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.