THDC: Training Hyperdimensional Computing Models with Backpropagation

• URL: http://arxiv.org/abs/2602.00116v1
• Date: Tue, 27 Jan 2026 10:33:36 GMT
• Title: THDC: Training Hyperdimensional Computing Models with Backpropagation
• Authors: Hanne Dejonghe, Sam Leroux,
• Abstract summary: Hyperdimensional computing (HDC) offers lightweight learning for energy-constrained devices by encoding data into high-dimensional vectors.<n>We propose Trainable Hyperdimensional Computing (THDC), which enables end-to-end HDC via backpropagation.<n>THDC replaces randomly vectors with trainable embeddings and introduces a one-layer binary neural network to optimize class representations.
• Score: 5.013248430919224
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Hyperdimensional computing (HDC) offers lightweight learning for energy-constrained devices by encoding data into high-dimensional vectors. However, its reliance on ultra-high dimensionality and static, randomly initialized hypervectors limits memory efficiency and learning capacity. Therefore, we propose Trainable Hyperdimensional Computing (THDC), which enables end-to-end HDC via backpropagation. THDC replaces randomly initialized vectors with trainable embeddings and introduces a one-layer binary neural network to optimize class representations. Evaluated on MNIST, Fashion-MNIST and CIFAR-10, THDC achieves equal or better accuracy than state-of-the-art HDC, with dimensionality reduced from 10.000 to 64.

Related papers

• Protocol Models: Scaling Decentralized Training with Communication-Efficient Model Parallelism [59.79227116582264]
  Scaling models has led to significant advancements in deep learning, but training these models in decentralized settings remains challenging.<n>We propose a novel compression algorithm that compresses both forward and backward passes, enabling up to 99% compression with no convergence degradation.
  arXiv  Detail & Related papers  (2025-06-02T02:19:22Z)
• FFT-based Dynamic Subspace Selection for Low-Rank Adaptive Optimization of Large Language Models [49.397861654088636]
  We propose a two-step procedure to approximate SVD/QR-based gradient projections into lower-dimensional spaces.<n>We show that our strategy achieves faster runtime and reduced memory usage by up to $25%$ across different model sizes.
  arXiv  Detail & Related papers  (2025-05-23T14:37:00Z)
• Transolver++: An Accurate Neural Solver for PDEs on Million-Scale Geometries [67.63077028746191]
  Transolver++ is a highly parallel and efficient neural solver that can solve PDEs on million-scale geometries.<n>Transolver++ increases the single- GPU input capacity to million-scale points for the first time.<n>It achieves over 20% performance gain in million-scale high-fidelity industrial simulations.
  arXiv  Detail & Related papers  (2025-02-04T15:33:50Z)
• TensorGRaD: Tensor Gradient Robust Decomposition for Memory-Efficient Neural Operator Training [91.8932638236073]
  We introduce textbfTensorGRaD, a novel method that directly addresses the memory challenges associated with large-structured weights.<n>We show that sparseGRaD reduces total memory usage by over $50%$ while maintaining and sometimes even improving accuracy.
  arXiv  Detail & Related papers  (2025-01-04T20:51:51Z)
• uHD: Unary Processing for Lightweight and Dynamic Hyperdimensional Computing [1.7118124088316602]
  Hyperdimensional computing (HDC) is a novel computational paradigm that operates on long-dimensional vectors known as hypervectors. In this paper, we show how to generate intensity and position hypervectors in HDC using low-discrepancy sequences. For the first time in the literature, our proposed approach employs lightweight vector generators utilizing unary bit-streams for efficient encoding of data.
  arXiv  Detail & Related papers  (2023-11-16T06:28:19Z)
• Efficient Hyperdimensional Computing [4.8915861089531205]
  We develop HDC models that use binary hypervectors with dimensions orders of magnitude lower than those of state-of-the-art HDC models. For instance, on the MNIST dataset, we achieve 91.12% HDC accuracy in image classification with a dimension of only 64.
  arXiv  Detail & Related papers  (2023-01-26T02:22:46Z)
• Towards Theoretically Inspired Neural Initialization Optimization [66.04735385415427]
  We propose a differentiable quantity, named GradCosine, with theoretical insights to evaluate the initial state of a neural network. We show that both the training and test performance of a network can be improved by maximizing GradCosine under norm constraint. Generalized from the sample-wise analysis into the real batch setting, NIO is able to automatically look for a better initialization with negligible cost.
  arXiv  Detail & Related papers  (2022-10-12T06:49:16Z)
• An Extension to Basis-Hypervectors for Learning from Circular Data in Hyperdimensional Computing [62.997667081978825]
  Hyperdimensional Computing (HDC) is a computation framework based on properties of high-dimensional random spaces. We present a study on basis-hypervector sets, which leads to practical contributions to HDC in general. We introduce a method to learn from circular data, an important type of information never before addressed in machine learning with HDC.
  arXiv  Detail & Related papers  (2022-05-16T18:04:55Z)
• EnHDC: Ensemble Learning for Brain-Inspired Hyperdimensional Computing [2.7462881838152913]
  This paper presents the first effort in exploring ensemble learning in the context of hyperdimensional computing. We propose the first ensemble HDC model referred to as EnHDC. We show that EnHDC can achieve on average 3.2% accuracy improvement over a single HDC classifier.
  arXiv  Detail & Related papers  (2022-03-25T09:54:00Z)
• A Brain-Inspired Low-Dimensional Computing Classifier for Inference on Tiny Devices [17.976792694929063]
  We propose a low-dimensional computing (LDC) alternative to hyperdimensional computing (HDC) We map our LDC classifier into a neural equivalent network and optimize our model using a principled training approach. Our LDC classifier offers an overwhelming advantage over the existing brain-inspired HDC models and is particularly suitable for inference on tiny devices.
  arXiv  Detail & Related papers  (2022-03-09T17:20:12Z)
• AC/DC: Alternating Compressed/DeCompressed Training of Deep Neural Networks [78.62086125399831]
  We present a general approach called Alternating Compressed/DeCompressed (AC/DC) training of deep neural networks (DNNs) AC/DC outperforms existing sparse training methods in accuracy at similar computational budgets. An important property of AC/DC is that it allows co-training of dense and sparse models, yielding accurate sparse-dense model pairs at the end of the training process.
  arXiv  Detail & Related papers  (2021-06-23T13:23:00Z)
• Classification using Hyperdimensional Computing: A Review [16.329917143918028]
  This paper introduces the background of HD computing, and reviews the data representation, data transformation, and similarity measurement. Evaluations indicate that HD computing shows great potential in addressing problems using data in the form of letters, signals and images.
  arXiv  Detail & Related papers  (2020-04-19T23:51:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.