LogHD: Robust Compression of Hyperdimensional Classifiers via Logarithmic Class-Axis Reduction

• URL: http://arxiv.org/abs/2511.03938v1
• Date: Thu, 06 Nov 2025 00:33:21 GMT
• Title: LogHD: Robust Compression of Hyperdimensional Classifiers via Logarithmic Class-Axis Reduction
• Abstract summary: We introduce LogHD, a logarithmic class-axis reduction that replaces the $C$ per-class prototypes with $n!approx!lceillog_k.<n>Across datasets and injected bit flips, LogHD attains competitive accuracy with smaller models and higher resilience at matched memory.
• Score: 12.477312277548497
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Hyperdimensional computing (HDC) suits memory, energy, and reliability-constrained systems, yet the standard "one prototype per class" design requires $O(CD)$ memory (with $C$ classes and dimensionality $D$). Prior compaction reduces $D$ (feature axis), improving storage/compute but weakening robustness. We introduce LogHD, a logarithmic class-axis reduction that replaces the $C$ per-class prototypes with $n\!\approx\!\lceil\log_k C\rceil$ bundle hypervectors (alphabet size $k$) and decodes in an $n$-dimensional activation space, cutting memory to $O(D\log_k C)$ while preserving $D$. LogHD uses a capacity-aware codebook and profile-based decoding, and composes with feature-axis sparsification. Across datasets and injected bit flips, LogHD attains competitive accuracy with smaller models and higher resilience at matched memory. Under equal memory, it sustains target accuracy at roughly $2.5$-$3.0\times$ higher bit-flip rates than feature-axis compression; an ASIC instantiation delivers $498\times$ energy efficiency and $62.6\times$ speedup over an AMD Ryzen 9 9950X and $24.3\times$/$6.58\times$ over an NVIDIA RTX 4090, and is $4.06\times$ more energy-efficient and $2.19\times$ faster than a feature-axis HDC ASIC baseline.

Related papers

• Maglev: Sliding Recurrent Memory [8.788753986099902]
  ours is a recurrent Transformer architecture with fixed-size memory that generalizes sliding-window attention.<n>We train ours with a memory consistency loss that aligns $m_t$ with $m'_t$, allowing inference to use $P$ alone.
  arXiv  Detail & Related papers  (2026-08-03T20:40:49Z)
• RiT: Vanilla Diffusion Transformers Suffice in Representation Space [12.711808725422108]
  Flow matching with $x$prediction is known to exploit low-dimensional manifold structure effectively in pixel space citeli2025back.<n>We ask whether a pretrained representation space, while containing a low-dimensional data manifold of comparable intrinsic dimensionality, offers a distribution more favorable for flow-matching learning.
  arXiv  Detail & Related papers  (2026-05-21T04:21:43Z)
• SURGE: SuperBatch Unified Resource-efficient GPU Encoding for Heterogeneous Partitioned Data [3.1624024957575982]
  SURGE is a streaming encoding system deployed in production to generate embeddings for over 800 million texts across 40,000 logical partitions.<n>On 10M texts with 4 NVIDIA L4768, SURGE delivers 26,413 texts/s -- matching fixed-times throughput using 12.6times less memory.
  arXiv  Detail & Related papers  (2026-05-01T19:51:50Z)
• ZenBrain: A Neuroscience-Inspired 7-Layer Memory Architecture for Autonomous AI Systems [51.56484100374058]
  On LongMemEval-500, ZenBrain matches a long-context oracle's binary-judge accuracy to within 4.5 pp.<n> ZenBrain is a 7-layer neuroscience-inspired memory architecture.
  arXiv  Detail & Related papers  (2026-04-26T20:39:19Z)
• Scaling Attention via Feature Sparsity [50.64995497733461]
  Scaling Transformers to ultra-long contexts is bottlenecked by the $O(n2 d)$ cost of self-attention.<n>We propose Sparse Feature Attention, where queries and keys are represented as $k$-sparse codes that preserve high-dimensional expressivity.<n>Across GPT-2 and Qwen3 pretraining, SFA matches dense baselines while improving speed by up to $2.5times$ and reducing FLOPs and KV-cache by nearly 50%.
  arXiv  Detail & Related papers  (2026-03-17T08:41:50Z)
• LoRIF: Low-Rank Influence Functions for Scalable Training Data Attribution [62.830878652285406]
  Training data attribution identifies which training examples most influenced a model's prediction.<n>LoRIF exploits low-rank structures of gradient to address both bottlenecks.<n>On models from 0.1B to 70B parameters trained on datasets with millions of examples, LoRIF achieves up to 20$times$ storage reduction and query-time speedup.
  arXiv  Detail & Related papers  (2026-01-29T16:18:34Z)
• Evolution Strategies at the Hyperscale [57.75314521465674]
  We introduce EGGROLL, an evolution strategies (ES) algorithm designed to scale backprop-free optimization to large population sizes.<n>ES is a set of powerful blackbox optimisation methods that can handle non-differentiable or noisy objectives.<n>EGGROLL overcomes these bottlenecks by generating random matrices $Ain mathbbRmtimes r, Bin mathbbRntimes r$ with $rll min(m,n)$ to form a low-rank matrix perturbation $A Btop$
  arXiv  Detail & Related papers  (2025-11-20T18:56:05Z)
• DecoHD: Decomposed Hyperdimensional Classification under Extreme Memory Budgets [8.226639797032483]
  Decomposition is a proven way to shrink deep networks without changing I/O.<n>We introduce DecoHD, which learns directly in a decomposed HDC parameterization.<n>In evaluation, DecoHD attains extreme memory savings with only minor degradation accuracy.
  arXiv  Detail & Related papers  (2025-11-05T23:28:02Z)
• Measurement-Based Fault-Tolerant Quantum Computation on High-Connectivity Devices: A Resource-Efficient Approach toward Early FTQC [2.511618458985613]
  We propose a measurement-based FTQC architecture for high-connectivity platforms such as trapped ions and neutral atoms.<n>Key idea is to use verified logical ancillas combined with Knill's error-correcting teleportation.<n>We present two implementations benchmarked under circuit-level depolarizing noise.
  arXiv  Detail & Related papers  (2025-10-21T14:02:20Z)
• Proving the Limited Scalability of Centralized Distributed Optimization via a New Lower Bound Construction [57.93371273485736]
  We consider a centralized distributed learning setup where all workers jointly find an unbiased bound LDeltaepsilon2,$ better poly-logarithmically in $n$, even in the homogeneous (i.i.d.) case, where all workers access the same distribution.
  arXiv  Detail & Related papers  (2025-06-30T13:27:39Z)
• Autoencoder-Based Hybrid Replay for Class-Incremental Learning [10.061328213032088]
  In class-incremental learning (CIL), effective incremental learning strategies are essential to mitigate task confusion and forgetting.<n>We propose an autoencoder-based hybrid replay (AHR) strategy that leverages our new hybrid autoencoder (HAE) to function as a compressor.
  arXiv  Detail & Related papers  (2025-05-09T09:59:12Z)
• 70% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float [52.079202872069835]
  Large-scale AI models, such as Large Language Models (LLMs) and Diffusion Models (DMs) have grown rapidly in size.<n>We introduce Dynamic-Length Float (DFloat11), a compression framework that reduces LLM and DM size by 30% while preserving outputs that are bit-for-bit identical to the original model.
  arXiv  Detail & Related papers  (2025-04-15T22:38:38Z)
• BurTorch: Revisiting Training from First Principles by Coupling Autodiff, Math Optimization, and Systems [56.16884466478886]
  BurTorch is a compact high-performance framework designed to optimize Deep Learning (DL) training on single-node workstations.<n>BurTorch adopts a minimalist design and demonstrates that, in these circumstances, classical compiled programming languages can play a significant role in DL research.
  arXiv  Detail & Related papers  (2025-03-18T00:52:12Z)
• Position-Aware Depth Decay Decoding ($D^3$): Boosting Large Language Model Inference Efficiency [26.173523821684306]
  A token-position aware layer skipping framework is proposed to save 1.5x times operations efficiently while maintaining performance.<n>Experiments on large language models with $7 sim 70$ billion parameters show that $D3$ can achieve an average 1.5x speedup compared with the full-inference pipeline.
  arXiv  Detail & Related papers  (2025-03-11T15:15:54Z)
• Hardware-Aware Parallel Prompt Decoding for Memory-Efficient Acceleration of LLM Inference [23.633481089469836]
  Auto-regressive decoding of Large Language Models (LLMs) results in significant overheads in their hardware performance.<n>We propose a novel parallel prompt decoding that requires only $0.0002$% trainable parameters, enabling efficient training on a single A100-40GB GPU in just 16 hours.<n>Our approach demonstrates up to 2.49$times$ speedup and maintains a minimal memory overhead of just $0.0004$%.
  arXiv  Detail & Related papers  (2024-05-28T22:19:30Z)
• H$_2$O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models [110.06476624089679]
  We introduce a novel approach for implementing the KV cache which significantly reduces its memory footprint. Our approach is based on the observation that a small portion of tokens contributes most of the value when computing attention scores. We propose Heavy Hitter (H$$O), a KV cache eviction policy that dynamically retains a balance of recent and H$$ tokens.
  arXiv  Detail & Related papers  (2023-06-24T20:11:14Z)
• HDCC: A Hyperdimensional Computing compiler for classification on embedded systems and high-performance computing [58.720142291102135]
  This work introduces the name compiler, the first open-source compiler that translates high-level descriptions of HDC classification methods into optimized C code. name is designed like a modern compiler, featuring an intuitive and descriptive input language, an intermediate representation (IR), and a retargetable backend. To substantiate these claims, we conducted experiments with HDCC on several of the most popular datasets in the HDC literature.
  arXiv  Detail & Related papers  (2023-04-24T19:16:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.