Memristive-Friendly Hadamard Reservoir Computing: Structured, Multiplier-Free Recurrences at Scale

• URL: http://arxiv.org/abs/2608.28295v1
• Date: Fri, 28 Aug 2026 12:58:42 GMT
• Title: Memristive-Friendly Hadamard Reservoir Computing: Structured, Multiplier-Free Recurrences at Scale
• Abstract summary: Reservoir Computing (RC) designs Recurrent Neural Networks around a fixed, untrained, recurrent layer, and is a natural candidate for neuromorphic hardware.<n>We replace the dense matrix with a structured orthogonal operator, built from sign diagonals, a permutation, and a fast Walsh-Hadamard transform.<n>We time the recurrent step on three hardware platforms, where it is up to $50times$ faster than a dense product and $104times$ smaller in memory.
• Score: 10.72107520133713
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Reservoir Computing (RC) designs Recurrent Neural Networks around a fixed, i.e., untrained, recurrent layer, and is a natural candidate for neuromorphic hardware. Memristive-friendly reservoirs derive the neuron dynamics from memristive-device kinetics, but still rely on dense recurrent matrices, which are expensive to realize physically. In this paper, we replace the dense matrix with a structured orthogonal operator, built from sign diagonals, a permutation, and a fast Walsh-Hadamard transform. The operator is multiplier-free, requires $O(N)$ parameters and $O(N\log N)$ operations per step, and is never materialized as a matrix. We instantiate it in a standard and in a memristive-friendly Echo State Network, with one binary input connection per unit. Our mathematical analysis shows that exact orthogonality yields an echo state condition that is tight in the recurrent scaling, and a noise response that is predictable at design time. Moreover, the operator mixes the whole state in a single application. Experiments on twenty classification and seven regression benchmarks, at reservoir sizes up to $N = 8192$, show that the structured models match dense orthogonal reservoirs, and achieve better mean performance than the cycle reservoir by a margin that widens with size. Furthermore, we time the recurrent step on three hardware platforms, where it is up to $50\times$ faster than a dense product and $10^4\times$ smaller in memory. Finally, we ablate the operator and measure the response to noise, quantization, device mismatch and discrete faults.

Related papers

• The Von-Neumann State-Space Transformer for neural decoding [0.0]
  Cortical computation is strikingly low-dimensional.<n>We introduce a memory-augmented Transformer whose feed-forward block is a low-rank instruction bank.<n>VN-SST is far more data-efficient than a modern Transformer.
  arXiv  Detail & Related papers  (2026-08-25T19:28:33Z)
• Attention at the Theoretical Minimum: A Mathematics of Arrays Framework for Memory-Optimal Transformer Kernels [0.0]
  We present a reformulation of scaled dot-product attention and its numerically stable softmax.<n>The DNF achieves $O(n_dk + n_dv)$ data movement versus $O(n2 + n_dk + n_dv)$ for the standard implementation.<n>Unlike hardware-specific accelerators or empirical tiling schemes such as FlashAttention, MoA simultaneously provides array fusion, shape-transformation correctness, and predictive cost models.
  arXiv  Detail & Related papers  (2026-06-05T14:44:49Z)
• The Recurrent Transformer: Greater Effective Depth and Efficient Decoding [48.9323408950142]
  We introduce the Recurrent Transformer, a simple architectural change where each layer attends to key-value pairs computed off its own activations.<n>We show that the architecture can emulate both (i) a conventional Transformer and (ii) token-to-gressive recurrent updates under mild assumptions.
  arXiv  Detail & Related papers  (2026-04-23T02:12:58Z)
• Evolution Strategies at the Hyperscale [57.75314521465674]
  We introduce EGGROLL, an evolution strategies (ES) algorithm designed to scale backprop-free optimization to large population sizes.<n>ES is a set of powerful blackbox optimisation methods that can handle non-differentiable or noisy objectives.<n>EGGROLL overcomes these bottlenecks by generating random matrices $Ain mathbbRmtimes r, Bin mathbbRntimes r$ with $rll min(m,n)$ to form a low-rank matrix perturbation $A Btop$
  arXiv  Detail & Related papers  (2025-11-20T18:56:05Z)
• Structured Sparse Transition Matrices to Enable State Tracking in State-Space Models [68.31088463716269]
  We propose a structured sparse parametrization of transition matrices in state-space models (SSMs)<n>Our method, PD-SSM, parametrizes the transition matrix as the product of a column one-hot matrix ($P$) and a complex-valued diagonal matrix ($D$)<n>The model significantly outperforms a wide collection of modern SSM variants on various FSA state tracking tasks.
  arXiv  Detail & Related papers  (2025-09-26T12:46:30Z)
• LREI: A fast numerical solver for quantum Landau-Lifshitz equations [0.0]
  We develop LREI, a memory- and time-efficient scheme for solving quantum Landau-Lifshitz (q-LL) and quantum Landau-Lifshitz-Gilbert (q-LLG) equations.
  arXiv  Detail & Related papers  (2025-08-28T20:33:37Z)
• Scaling Up Liquid-Resistance Liquid-Capacitance Networks for Efficient Sequence Modeling [50.994194925685434]
  LrcSSM is a $textitnon-linear$ recurrent model that processes long sequences as fast as today's linear state-space layers.<n>By forcing its Jacobian matrix to be diagonal, the full sequence can be solved in parallel.<n>LrcSSM offers a formal gradient-stability guarantee that other input-varying systems such as Liquid-S4 do not provide.
  arXiv  Detail & Related papers  (2025-05-27T20:02:59Z)
• Transformer Neural Processes - Kernel Regression [2.309018557701645]
  We introduce the Transformer Neural Process - Kernel Regression (TNP-KR), a scalable Neural Process (NP)<n>TNP-KR features a Kernel Regression Block (KR-Block), a simple, parameter, and efficient transformer block with complexity $O(n_c2 + n_c n_t)$, and two novel attention mechanisms: scan attention (SA), a memory-efficient scan-based bias, and deep kernel attention (DKA), a Performer-style attention that implicitly incoporates a distance bias.<n>These enhancements enable both TNP-KR variants to perform inference with 100K
  arXiv  Detail & Related papers  (2024-11-19T13:40:49Z)
• TCNCA: Temporal Convolution Network with Chunked Attention for Scalable Sequence Processing [52.64837396100988]
  MEGA is a recent transformer-based architecture, which utilizes a linear recurrent operator whose parallel computation, based on the FFT, scales as $O(LlogL)$, with $L$ being the sequence length. We build upon their approach by replacing the linear recurrence with a special temporal convolutional network which permits larger receptive field size with shallower networks, and reduces the computational complexity to $O(L)$. We evaluate TCNCA on EnWik8 language modeling, long-range-arena (LRA) sequence classification, as well as a synthetic reasoning benchmark associative recall.
  arXiv  Detail & Related papers  (2023-12-09T16:12:25Z)
• VersaGNN: a Versatile accelerator for Graph neural networks [81.1667080640009]
  We propose textitVersaGNN, an ultra-efficient, systolic-array-based versatile hardware accelerator. textitVersaGNN achieves on average 3712$times$ speedup with 1301.25$times$ energy reduction on CPU, and 35.4$times$ speedup with 17.66$times$ energy reduction on GPU.
  arXiv  Detail & Related papers  (2021-05-04T04:10:48Z)
• Constant-Depth and Subcubic-Size Threshold Circuits for Matrix Multiplication [1.9518237361775532]
  Recent advances in large-scale neural computing hardware has made their practical implementation a near-term possibility. We describe a theoretical approach for multiplying two $N$ by $N$ matrices that integrates threshold gate logic. Dense matrix multiplication is a core operation in convolutional neural network training.
  arXiv  Detail & Related papers  (2020-06-25T18:28:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.