Rank-Order N-of-M Codes for Sparse Distributed Memory: Disentangling Representation and Learning Effects in Noise Robustness Against Contemporary Neuromorphic Architectures

• URL: http://arxiv.org/abs/2607.02967v2
• Date: Tue, 07 Jul 2026 14:20:15 GMT
• Title: Rank-Order N-of-M Codes for Sparse Distributed Memory: Disentangling Representation and Learning Effects in Noise Robustness Against Contemporary Neuromorphic Architectures
• Abstract summary: This paper evaluates rank-order N-of-M encoding as an alternative to the threshold-binary encoder in CALM.<n> experiments on GloVe-100 embeddings confirm this small but consistent encoding benefit on real structured data.<n>Results identify which components of the original rank-order SDM architecture provide measurable benefits for contemporary memory-augmented AI systems.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models remain limited as continual learning systems, motivating renewed interest in Sparse Distributed Memory (SDM) as an explicit online episodic memory. CALM (Nechesov and Ruponen, 2025) identifies its threshold-binary encoder as an open design question. This paper evaluates rank-order N-of-M encoding (Furber et al., 2007) as an alternative. We make three contributions. First, a faithful reimplementation validates the published architecture by confirming exact equivalence between WheelSDM and RankOrderSDM (cosine similarity 1.0000 across 10 seeds) and reproducing the documented divergence of RDLIF neurons under interference. Second, multi-seed capacity experiments show RankOrderSDM outperforming StandardSDM by 13.4 percentage points at saturation in the scaled configuration and by 0.8 percentage points at the published architecture scale. Third, BER robustness experiments disentangle representation and learning effects, showing that the large robustness gain arises primarily from the interaction of rank-order encoding with MAX-Hebbian learning, while the encoder alone provides only a small advantage under matched learning conditions. Experiments on GloVe-100 embeddings confirm this small but consistent encoding benefit on real structured data, whereas sentence embeddings exhibit a ceiling effect at low memory load. A secondary analysis shows that idealized rank-order encoding requires half the component-level encoding energy of SpikingMamba's SI-LIF neurons at four-bit precision, although decoder costs dominate overall system energy. These results identify which components of the original rank-order SDM architecture provide measurable benefits for contemporary memory-augmented AI systems, offering practical guidance for architectures such as CALM.

Related papers

• Decoder-Free Supervoxel GNN for Accurate Brain-Tumor Localization in Multi-Modal MRI [1.209913077217557]
  SVGFormer is a decoder-free pipeline that partitions the volume into a semantic graph of supervoxels.<n>Its hierarchical encoder learns rich node representations by combining a patch-level Transformer with a supervoxel-level Graph Attention Network.<n>Our results establish that a graph-based, encoder-only paradigm offers an accurate and inherently interpretable alternative for 3D medical image representation.
  arXiv  Detail & Related papers  (2026-01-20T15:13:04Z)
• BrainCSD: A Hierarchical Consistency-Driven MoE Foundation Model for Unified Connectome Synthesis and Multitask Brain Trait Prediction [33.650792366699385]
  Functional and structural connectivity (FC/SC) are key biomarkers for brain analysis, yet their clinical utility is hindered by costly acquisition, complex preprocessing, and frequent missing modalities.<n>We propose BrainCSD, a hierarchical mixture-of-experts foundation model that jointly synthesizes FC/SC biomarkers and supports downstream decoding tasks (diagnosis and prediction)<n>BrainCSD achieves 95.6%% accuracy for MCI vs. CN classification without FC, low error synthesis (FC RMSE: 0.038; SC RMSE: 0.006), brain age prediction (MAE: 4.04 years), and MMSE score (MAE: 1.72 points
  arXiv  Detail & Related papers  (2025-11-07T04:40:47Z)
• Knowledge-Informed Neural Network for Complex-Valued SAR Image Recognition [51.03674130115878]
  We introduce the Knowledge-Informed Neural Network (KINN), a lightweight framework built upon a novel "compression-aggregation-compression" architecture.<n>KINN establishes a state-of-the-art in parameter-efficient recognition, offering exceptional generalization in data-scarce and out-of-distribution scenarios.
  arXiv  Detail & Related papers  (2025-10-23T07:12:26Z)
• Decoder-Hybrid-Decoder Architecture for Efficient Reasoning with Long Generation [108.0657508755532]
  We introduce the Gated Memory Unit (GMU), a simple yet effective mechanism for efficient memory sharing across layers.<n>We apply it to create SambaY, a decoder-hybrid-decoder architecture that incorporates GMUs to share memory readout states from a Samba-based self-decoder.
  arXiv  Detail & Related papers  (2025-07-09T07:27:00Z)
• Unified CNNs and transformers underlying learning mechanism reveals multi-head attention modus vivendi [0.0]
  Convolutional neural networks (CNNs) evaluate short-range correlations in input images which progress along the layers.<n> vision transformer (ViT) architectures evaluate long-range correlations, using repeated transformer encoders composed of fully connected layers.<n>This study demonstrates that CNNs and ViT architectures stem from a unified underlying learning mechanism.
  arXiv  Detail & Related papers  (2025-01-22T14:19:48Z)
• Heterogenous Memory Augmented Neural Networks [84.29338268789684]
  We introduce a novel heterogeneous memory augmentation approach for neural networks. By introducing learnable memory tokens with attention mechanism, we can effectively boost performance without huge computational overhead. We show our approach on various image and graph-based tasks under both in-distribution (ID) and out-of-distribution (OOD) conditions.
  arXiv  Detail & Related papers  (2023-10-17T01:05:28Z)
• RCMHA: Relative Convolutional Multi-Head Attention for Natural Language Modelling [0.0]
  Relative Multi-Head Attention (RMHA) has superior accuracy, boasting a score of 0.572 in comparison to alternative attention modules. RMHA emerges as the most frugal, demonstrating an average consumption of 2.98 GB, surpassing RMHA which necessitates 3.5 GB.
  arXiv  Detail & Related papers  (2023-08-07T09:24:24Z)
• Neural Attentive Circuits [93.95502541529115]
  We introduce a general purpose, yet modular neural architecture called Neural Attentive Circuits (NACs) NACs learn the parameterization and a sparse connectivity of neural modules without using domain knowledge. NACs achieve an 8x speedup at inference time while losing less than 3% performance.
  arXiv  Detail & Related papers  (2022-10-14T18:00:07Z)
• Neural Architecture Search For LF-MMI Trained Time Delay Neural Networks [61.76338096980383]
  A range of neural architecture search (NAS) techniques are used to automatically learn two types of hyper- parameters of state-of-the-art factored time delay neural networks (TDNNs) These include the DARTS method integrating architecture selection with lattice-free MMI (LF-MMI) TDNN training. Experiments conducted on a 300-hour Switchboard corpus suggest the auto-configured systems consistently outperform the baseline LF-MMI TDNN systems.
  arXiv  Detail & Related papers  (2020-07-17T08:32:11Z)
• When Residual Learning Meets Dense Aggregation: Rethinking the Aggregation of Deep Neural Networks [57.0502745301132]
  We propose Micro-Dense Nets, a novel architecture with global residual learning and local micro-dense aggregations. Our micro-dense block can be integrated with neural architecture search based models to boost their performance.
  arXiv  Detail & Related papers  (2020-04-19T08:34:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.