A Theoretical Perspective on Hyperdimensional Computing

• URL: http://arxiv.org/abs/2010.07426v3
• Date: Thu, 17 Feb 2022 23:29:29 GMT
• Title: A Theoretical Perspective on Hyperdimensional Computing
• Authors: Anthony Thomas, Sanjoy Dasgupta, Tajana Rosing
• Abstract summary: Hyperdimensional (HD) computing is a set of neurally inspired methods for obtaining high-dimensional, low-precision, distributed representations of data. HD computing has recently garnered significant interest from the computer hardware community as an energy-efficient, low-latency, and noise-robust tool for solving learning problems.
• Score: 17.50442191930551
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Hyperdimensional (HD) computing is a set of neurally inspired methods for obtaining high-dimensional, low-precision, distributed representations of data. These representations can be combined with simple, neurally plausible algorithms to effect a variety of information processing tasks. HD computing has recently garnered significant interest from the computer hardware community as an energy-efficient, low-latency, and noise-robust tool for solving learning problems. In this review, we present a unified treatment of the theoretical foundations of HD computing with a focus on the suitability of representations for learning.

Related papers

• Mechanistic Neural Networks for Scientific Machine Learning [58.99592521721158]
  We present Mechanistic Neural Networks, a neural network design for machine learning applications in the sciences. It incorporates a new Mechanistic Block in standard architectures to explicitly learn governing differential equations as representations. Central to our approach is a novel Relaxed Linear Programming solver (NeuRLP) inspired by a technique that reduces solving linear ODEs to solving linear programs.
  arXiv  Detail & Related papers  (2024-02-20T15:23:24Z)
• The Hyperdimensional Transform for Distributional Modelling, Regression and Classification [12.693238093510072]
  We present the power of the hyperdimensional transform to a broad data science audience. We show how existing algorithms can be modified and how this transform can lead to a novel, well-founded toolbox.
  arXiv  Detail & Related papers  (2023-11-14T13:26:49Z)
• Robust and Scalable Hyperdimensional Computing With Brain-Like Neural Adaptations [17.052624039805856]
  Internet of Things (IoT) has facilitated many applications utilizing edge-based machine learning (ML) methods to analyze locally collected data. Brain-inspired hyperdimensional computing (HDC) has been introduced to address this issue. Existing HDCs use static encoders, requiring extremely high dimensionality and hundreds of training iterations to achieve reasonable accuracy. We present dynamic HDC learning frameworks that identify and regenerate undesired dimensions to provide adequate accuracy with significantly lowered dimensionalities.
  arXiv  Detail & Related papers  (2023-11-13T19:42:33Z)
• Brain-Inspired Computational Intelligence via Predictive Coding [89.6335791546526]
  Predictive coding (PC) has shown promising performance in machine intelligence tasks. PC can model information processing in different brain areas, can be used in cognitive control and robotics.
  arXiv  Detail & Related papers  (2023-08-15T16:37:16Z)
• Rotating Features for Object Discovery [74.1465486264609]
  We present Rotating Features, a generalization of complex-valued features to higher dimensions, and a new evaluation procedure for extracting objects from distributed representations. Together, these advancements enable us to scale distributed object-centric representations from simple toy to real-world data.
  arXiv  Detail & Related papers  (2023-06-01T12:16:26Z)
• Streaming Encoding Algorithms for Scalable Hyperdimensional Computing [12.829102171258882]
  Hyperdimensional computing (HDC) is a paradigm for data representation and learning originating in computational neuroscience. In this work, we explore a family of streaming encoding techniques based on hashing. We show formally that these methods enjoy comparable guarantees on performance for learning applications while being substantially more efficient than existing alternatives.
  arXiv  Detail & Related papers  (2022-09-20T17:25:14Z)
• Inducing Gaussian Process Networks [80.40892394020797]
  We propose inducing Gaussian process networks (IGN), a simple framework for simultaneously learning the feature space as well as the inducing points. The inducing points, in particular, are learned directly in the feature space, enabling a seamless representation of complex structured domains. We report on experimental results for real-world data sets showing that IGNs provide significant advances over state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-04-21T05:27:09Z)
• Neuromorphic scaling advantages for energy-efficient random walk computation [0.28144129864580447]
  Neuromorphic computing aims to replicate the brain's computational structure and architecture in man-made hardware. We show that high-degree parallelism and configurability of spiking neuromorphic architectures makes them well-suited to implement random walks via discrete time chains. We find that NMC platforms, at a sufficient scale, can drastically reduce the energy demands of high-performance computing platforms.
  arXiv  Detail & Related papers  (2021-07-27T19:44:33Z)
• Tensor Methods in Computer Vision and Deep Learning [120.3881619902096]
  tensors, or multidimensional arrays, are data structures that can naturally represent visual data of multiple dimensions. With the advent of the deep learning paradigm shift in computer vision, tensors have become even more fundamental. This article provides an in-depth and practical review of tensors and tensor methods in the context of representation learning and deep learning.
  arXiv  Detail & Related papers  (2021-07-07T18:42:45Z)
• Spiking Neural Networks Hardware Implementations and Challenges: a Survey [53.429871539789445]
  Spiking Neural Networks are cognitive algorithms mimicking neuron and synapse operational principles. We present the state of the art of hardware implementations of spiking neural networks. We discuss the strategies employed to leverage the characteristics of these event-driven algorithms at the hardware level.
  arXiv  Detail & Related papers  (2020-05-04T13:24:00Z)
• Classification using Hyperdimensional Computing: A Review [16.329917143918028]
  This paper introduces the background of HD computing, and reviews the data representation, data transformation, and similarity measurement. Evaluations indicate that HD computing shows great potential in addressing problems using data in the form of letters, signals and images.
  arXiv  Detail & Related papers  (2020-04-19T23:51:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.