DecoHD: Decomposed Hyperdimensional Classification under Extreme Memory Budgets

• URL: http://arxiv.org/abs/2511.03911v1
• Date: Wed, 05 Nov 2025 23:28:02 GMT
• Title: DecoHD: Decomposed Hyperdimensional Classification under Extreme Memory Budgets
• Abstract summary: Decomposition is a proven way to shrink deep networks without changing I/O.<n>We introduce DecoHD, which learns directly in a decomposed HDC parameterization.<n>In evaluation, DecoHD attains extreme memory savings with only minor degradation accuracy.
• Score: 8.226639797032483
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Decomposition is a proven way to shrink deep networks without changing I/O. We bring this idea to hyperdimensional computing (HDC), where footprint cuts usually shrink the feature axis and erode concentration and robustness. Prior HDC decompositions decode via fixed atomic hypervectors, which are ill-suited for compressing learned class prototypes. We introduce DecoHD, which learns directly in a decomposed HDC parameterization: a small, shared set of per-layer channels with multiplicative binding across layers and bundling at the end, yielding a large representational space from compact factors. DecoHD compresses along the class axis via a lightweight bundling head while preserving native bind-bundle-score; training is end-to-end, and inference remains pure HDC, aligning with in/near-memory accelerators. In evaluation, DecoHD attains extreme memory savings with only minor accuracy degradation under tight deployment budgets. On average it stays within about 0.1-0.15% of a strong non-reduced HDC baseline (worst case 5.7%), is more robust to random bit-flip noise, reaches its accuracy plateau with up to ~97% fewer trainable parameters, and -- in hardware -- delivers roughly 277x/35x energy/speed gains over a CPU (AMD Ryzen 9 9950X), 13.5x/3.7x over a GPU (NVIDIA RTX 4090), and 2.0x/2.4x over a baseline HDC ASIC.

Related papers

• FlashMemory-DeepSeek-V4: Lightning Index Ultra-Long Context via Lookahead Sparse Attention [77.12062766962815]
  Lookahead Sparse Attention (LSA) is powered by a Neural Memory Indexer built upon the DeepSeek-V4 architecture.<n>We instantiate this architecture via a backbone-free decoupled training strategy.<n>We show that FM-DS-V4 compresses the average physical KV cache footprint down to merely 13.5% of the full-context baseline.
  arXiv  Detail & Related papers  (2026-06-08T06:25:54Z)
• Scaling Parallel Sequence Models to Foundation-Scale Vision Encoders [123.58723804218151]
  Vision foundation models are bottlenecked by the quadratic cost of self-attention.<n>We present C-GSPN based on 2D spatial propagation propagation.
  arXiv  Detail & Related papers  (2026-05-30T14:29:43Z)
• ImageHD: Energy-Efficient On-Device Continual Learning of Visual Representations via Hyperdimensional Computing [0.07646713951724012]
  ImageHD is an FPGA accelerator for on-device continual learning of visual data based on HDC.<n>On CORe50, ImageHD achieves up to 40.4x (4.84x) speedup and 383x (105.1x) energy efficiency over optimized CPU ( GPU) baselines.
  arXiv  Detail & Related papers  (2026-04-23T04:55:18Z)
• TeCoNeRV: Leveraging Temporal Coherence for Compressible Neural Representations for Videos [51.99176811574457]
  Implicit Neural Representations (INRs) have recently demonstrated impressive performance for video compression.<n>However, scaling to high-resolution videos while maintaining encoding efficiency remains a significant challenge.<n>We address these fundamental limitations through three key contributions.<n>We are the first hypernetwork approach to demonstrate results at 480p, 720p and 1080p on UVG, HEVC and MCL-JCV.
  arXiv  Detail & Related papers  (2026-02-18T18:59:55Z)
• LogHD: Robust Compression of Hyperdimensional Classifiers via Logarithmic Class-Axis Reduction [12.477312277548497]
  We introduce LogHD, a logarithmic class-axis reduction that replaces the $C$ per-class prototypes with $n!approx!lceillog_k.<n>Across datasets and injected bit flips, LogHD attains competitive accuracy with smaller models and higher resilience at matched memory.
  arXiv  Detail & Related papers  (2025-11-06T00:33:21Z)
• CommVQ: Commutative Vector Quantization for KV Cache Compression [50.37946553931796]
  We propose Commutative Vector Quantization (CommVQ) to significantly reduce memory usage for long-context LLM inference.<n>We first introduce additive quantization with a lightweight encoder and codebook to compress the KV cache.<n>Our approach achieves high accuracy with additive quantization and low overhead via the RoPE-commutative codebook.
  arXiv  Detail & Related papers  (2025-06-23T17:50:11Z)
• ReCalKV: Low-Rank KV Cache Compression via Head Reordering and Offline Calibration [69.57122277845293]
  We propose ReCalKV, a post-training low-rank KV cache compression approach with tailored strategies for Keys and Values.<n>For Keys, we propose Similarity aware Recontext (HSR), which clusters structurally similar heads into groups, enabling more accurate low-rank approximation.<n>For Values, we propose Offline Head-wise Value (OVC), which efficiently calibrates the value projection matrix using calibration data without training.
  arXiv  Detail & Related papers  (2025-05-30T08:49:27Z)
• DPQ-HD: Post-Training Compression for Ultra-Low Power Hyperdimensional Computing [6.378578005171813]
  We propose a novel Post Training Compression algorithm, Decomposition-Pruning-Quantization (DPQ-HD)<n>DPQ-HD reduces computational and memory overhead by uniquely combining the above three compression techniques.<n>We demonstrate that DPQ-HD achieves up to 20-100x reduction in memory for image and graph classification tasks with only a 1-2% drop in accuracy.
  arXiv  Detail & Related papers  (2025-05-08T16:54:48Z)
• 70% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float [52.079202872069835]
  Large-scale AI models, such as Large Language Models (LLMs) and Diffusion Models (DMs) have grown rapidly in size.<n>We introduce Dynamic-Length Float (DFloat11), a compression framework that reduces LLM and DM size by 30% while preserving outputs that are bit-for-bit identical to the original model.
  arXiv  Detail & Related papers  (2025-04-15T22:38:38Z)
• LoCo: Low-Bit Communication Adaptor for Large-scale Model Training [63.040522637816906]
  Low-bit communication often degrades training quality due to compression information loss.<n>We propose Low-bit Communication Adaptor (LoCo), which compensates local local GPU nodes before, without compromising quality.<n> Experimental results show that across moving large-scale training model frameworks like Megatron-LM and PyTorchs FSDP, LoCo significantly improves compression communication efficiency.
  arXiv  Detail & Related papers  (2024-07-05T13:01:36Z)
• HDCC: A Hyperdimensional Computing compiler for classification on embedded systems and high-performance computing [58.720142291102135]
  This work introduces the name compiler, the first open-source compiler that translates high-level descriptions of HDC classification methods into optimized C code. name is designed like a modern compiler, featuring an intuitive and descriptive input language, an intermediate representation (IR), and a retargetable backend. To substantiate these claims, we conducted experiments with HDCC on several of the most popular datasets in the HDC literature.
  arXiv  Detail & Related papers  (2023-04-24T19:16:03Z)
• Efficient Hyperdimensional Computing [4.8915861089531205]
  We develop HDC models that use binary hypervectors with dimensions orders of magnitude lower than those of state-of-the-art HDC models. For instance, on the MNIST dataset, we achieve 91.12% HDC accuracy in image classification with a dimension of only 64.
  arXiv  Detail & Related papers  (2023-01-26T02:22:46Z)
• HDTorch: Accelerating Hyperdimensional Computing with GP-GPUs for Design Space Exploration [4.783565770657063]
  We introduce HDTorch, an open-source, PyTorch-based HDC library with extensions for hypervector operations. We analyze four HDC benchmark datasets in terms of accuracy, runtime, and memory consumption. We perform the first-ever HD training and inference analysis of the entirety of the CHB-MIT EEG epilepsy database.
  arXiv  Detail & Related papers  (2022-06-09T19:46:08Z)
• EnHDC: Ensemble Learning for Brain-Inspired Hyperdimensional Computing [2.7462881838152913]
  This paper presents the first effort in exploring ensemble learning in the context of hyperdimensional computing. We propose the first ensemble HDC model referred to as EnHDC. We show that EnHDC can achieve on average 3.2% accuracy improvement over a single HDC classifier.
  arXiv  Detail & Related papers  (2022-03-25T09:54:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.