Real-Valued Hyperdimensional Sequence Representations with Hadamard Product Binding and Shift Equivariance

• URL: http://arxiv.org/abs/2608.28334v1
• Date: Fri, 28 Aug 2026 13:45:37 GMT
• Title: Real-Valued Hyperdimensional Sequence Representations with Hadamard Product Binding and Shift Equivariance
• Abstract summary: This paper develops real-valued position encoding motivated by Random Features.<n>It aims to retain the desirable properties of Fractional Power representations while supporting Hadamard-based operations.<n>Experiments on time-series classification datasets show that the proposed real-valued representations achieve performance comparable to standard Fractional Power kernels.
• Score: 6.771283097297353
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Encoding temporal order is a fundamental requirement for sequence representations in Hyperdimensional Computing. Fractional Power Encoding provides similarity-preserving position vectors whose inner products approximate shift-invariant kernels, and it supports shift-equivariant transformations of encoded sequence representations. However, standard formulations of Fractional Power Encoding are primarily designed for binding operations such as circular convolution or complex-valued multiplication, which limits their compatibility with Hadamard product binding of real-valued vectors. This paper develops real-valued position encodings motivated by Random Fourier Features, aiming to retain the desirable properties of Fractional Power Encoding while supporting Hadamard-based operations. We propose three real-valued position-encoding variants: a real-valued baseline based on the inverse Fourier transform, and Sinusoid and Cosine-only representations derived from Random Fourier Features. Among them, the Sinusoid variant provides an explicit algebraic shift operator, allowing temporal shifts to be applied directly to the vector-encoded sequence representation without re-encoding the shifted sequence. Experiments on time-series classification datasets show that the proposed real-valued representations achieve performance comparable to standard Fractional Power Encoding while enabling computationally efficient Hadamard product binding. The Sinusoid variant offers the most favorable trade-off, combining efficient real-valued implementation with exact shift-equivariant transformations.

Related papers

• Spiking Sequence Machines and Transformers [0.0]
  Sequence learning reduces to similarity-based retrieval over a temporally indexed representation space.<n>We formalise a Phase-Late Isomorphism showing that sinusoidal positional phase and spike timing are linearly related.
  arXiv  Detail & Related papers  (2026-05-01T13:45:15Z)
• Weights to Code: Extracting Interpretable Algorithms from the Discrete Transformer [65.38883376379812]
  We propose the Discrete Transformer, an architecture engineered to bridge the gap between continuous representations and discrete symbolic logic.<n> Empirically, the Discrete Transformer not only achieves performance comparable to RNN-based baselines but crucially extends interpretability to continuous variable domains.
  arXiv  Detail & Related papers  (2026-01-09T12:49:41Z)
• WUSH: Near-Optimal Adaptive Transforms for LLM Quantization [52.77441224845925]
  Quantization to low bitwidth is a standard approach for deploying large language models.<n>A few extreme weights and activations stretch the dynamic range and reduce the effective resolution of the quantizer.<n>We derive, for the first time, closed-form optimal linear blockwise transforms for joint weight-activation quantization.
  arXiv  Detail & Related papers  (2025-11-30T16:17:34Z)
• Theoretical Analysis of Positional Encodings in Transformer Models: Impact on Expressiveness and Generalization [10.034655199520168]
  Positional encodings are a core part of transformer-based models.<n>This paper analyzes how various positional encoding methods impact a transformer's expressiveness, generalization ability, and extrapolation to longer sequences.
  arXiv  Detail & Related papers  (2025-06-05T23:02:18Z)
• Converting Transformers into DGNNs Form [3.7468283401703797]
  We introduce a synthetic unitary digraph convolution based on the digraph Fourier transform.<n>The resulting model, which we term Converter, effectively converts a Transformer into a Directed Graph Neural Network form.<n>We have tested Converter on Long-Range Arena benchmark, long document classification, and DNA sequence-based taxonomy classification.
  arXiv  Detail & Related papers  (2025-02-01T22:44:46Z)
• Variable-size Symmetry-based Graph Fourier Transforms for image compression [65.7352685872625]
  We propose a new family of Symmetry-based Graph Fourier Transforms of variable sizes into a coding framework. Our proposed algorithm generates symmetric graphs on the grid by adding specific symmetrical connections between nodes. Experiments show that SBGFTs outperform the primary transforms integrated in the explicit Multiple Transform Selection.
  arXiv  Detail & Related papers  (2024-11-24T13:00:44Z)
• PRformer: Pyramidal Recurrent Transformer for Multivariate Time Series Forecasting [82.03373838627606]
  Self-attention mechanism in Transformer architecture requires positional embeddings to encode temporal order in time series prediction. We argue that this reliance on positional embeddings restricts the Transformer's ability to effectively represent temporal sequences. We present a model integrating PRE with a standard Transformer encoder, demonstrating state-of-the-art performance on various real-world datasets.
  arXiv  Detail & Related papers  (2024-08-20T01:56:07Z)
• EulerFormer: Sequential User Behavior Modeling with Complex Vector Attention [88.45459681677369]
  We propose a novel transformer variant with complex vector attention, named EulerFormer. It provides a unified theoretical framework to formulate both semantic difference and positional difference. It is more robust to semantic variations and possesses moresuperior theoretical properties in principle.
  arXiv  Detail & Related papers  (2024-03-26T14:18:43Z)
• Addressing Token Uniformity in Transformers via Singular Value Transformation [24.039280291845706]
  Token uniformity is commonly observed in transformer-based models. We show that a less skewed singular value distribution can alleviate the token uniformity' problem.
  arXiv  Detail & Related papers  (2022-08-24T22:44:09Z)
• OneDConv: Generalized Convolution For Transform-Invariant Representation [76.15687106423859]
  We propose a novel generalized one dimension convolutional operator (OneDConv) It dynamically transforms the convolution kernels based on the input features in a computationally and parametrically efficient manner. It improves the robustness and generalization of convolution without sacrificing the performance on common images.
  arXiv  Detail & Related papers  (2022-01-15T07:44:44Z)
• Demystifying the Better Performance of Position Encoding Variants for Transformer [12.503079503907989]
  We show how to encode position and segment into Transformer models. The proposed method performs on par with SOTA on GLUE, XTREME and WMT benchmarks while saving costs.
  arXiv  Detail & Related papers  (2021-04-18T03:44:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.