Learning with Holographic Reduced Representations

• URL: http://arxiv.org/abs/2109.02157v1
• Date: Sun, 5 Sep 2021 19:37:34 GMT
• Title: Learning with Holographic Reduced Representations
• Authors: Ashwinkumar Ganesan, Hang Gao, Sunil Gandhi, Edward Raff, Tim Oates, James Holt, Mark McLean
• Abstract summary: Holographic Reduced Representations (HRR) are a method for performing symbolic AI on top of real-valued vectors. This paper revisits this approach to understand if it is viable for enabling a hybrid neural-symbolic approach to learning.
• Score: 28.462635977110413
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Holographic Reduced Representations (HRR) are a method for performing symbolic AI on top of real-valued vectors \cite{Plate1995} by associating each vector with an abstract concept, and providing mathematical operations to manipulate vectors as if they were classic symbolic objects. This method has seen little use outside of older symbolic AI work and cognitive science. Our goal is to revisit this approach to understand if it is viable for enabling a hybrid neural-symbolic approach to learning as a differentiable component of a deep learning architecture. HRRs today are not effective in a differentiable solution due to numerical instability, a problem we solve by introducing a projection step that forces the vectors to exist in a well behaved point in space. In doing so we improve the concept retrieval efficacy of HRRs by over $100\times$. Using multi-label classification we demonstrate how to leverage the symbolic HRR properties to develop an output layer and loss function that is able to learn effectively, and allows us to investigate some of the pros and cons of an HRR neuro-symbolic learning approach.

Related papers

• Differentiable Logic Programming for Distant Supervision [4.820391833117535]
  We introduce a new method for integrating neural networks with logic programming in Neural-Symbolic AI (NeSy) Unlike prior methods, our approach does not depend on symbolic solvers for reasoning about missing labels. This method facilitates more efficient learning under distant supervision.
  arXiv  Detail & Related papers  (2024-08-22T17:55:52Z)
• LARS-VSA: A Vector Symbolic Architecture For Learning with Abstract Rules [1.3049516752695616]
  We propose a "relational bottleneck" that separates object-level features from abstract rules, allowing learning from limited amounts of data. We adapt the "relational bottleneck" strategy to a high-dimensional space, incorporating explicit vector binding operations between symbols and relational representations. Our system benefits from the low overhead of operations in hyperdimensional space, making it significantly more efficient than the state of the art when evaluated on a variety of test datasets.
  arXiv  Detail & Related papers  (2024-05-23T11:05:42Z)
• A Novel Neural-symbolic System under Statistical Relational Learning [50.747658038910565]
  We propose a general bi-level probabilistic graphical reasoning framework called GBPGR. In GBPGR, the results of symbolic reasoning are utilized to refine and correct the predictions made by the deep learning models. Our approach achieves high performance and exhibits effective generalization in both transductive and inductive tasks.
  arXiv  Detail & Related papers  (2023-09-16T09:15:37Z)
• Goal Space Abstraction in Hierarchical Reinforcement Learning via Set-Based Reachability Analysis [0.5409704301731713]
  We introduce a Feudal HRL algorithm that concurrently learns both the goal representation and a hierarchical policy. We evaluate our approach on complex navigation tasks, showing the learned representation is interpretable, transferrable and results in data efficient learning.
  arXiv  Detail & Related papers  (2023-09-14T12:39:26Z)
• Semantic Graph Representation Learning for Handwritten Mathematical Expression Recognition [57.60390958736775]
  We propose a simple but efficient method to enhance semantic interaction learning (SIL) We first construct a semantic graph based on the statistical symbol co-occurrence probabilities. Then we design a semantic aware module (SAM), which projects the visual and classification feature into semantic space. Our method achieves better recognition performance than prior arts on both CROHME and HME100K datasets.
  arXiv  Detail & Related papers  (2023-08-21T06:23:41Z)
• Improving Deep Representation Learning via Auxiliary Learnable Target Coding [69.79343510578877]
  This paper introduces a novel learnable target coding as an auxiliary regularization of deep representation learning. Specifically, a margin-based triplet loss and a correlation consistency loss on the proposed target codes are designed to encourage more discriminative representations.
  arXiv  Detail & Related papers  (2023-05-30T01:38:54Z)
• Symbolic Visual Reinforcement Learning: A Scalable Framework with Object-Level Abstraction and Differentiable Expression Search [63.3745291252038]
  We propose DiffSES, a novel symbolic learning approach that discovers discrete symbolic policies. By using object-level abstractions instead of raw pixel-level inputs, DiffSES is able to leverage the simplicity and scalability advantages of symbolic expressions. Our experiments demonstrate that DiffSES is able to generate symbolic policies that are simpler and more scalable than state-of-the-art symbolic RL methods.
  arXiv  Detail & Related papers  (2022-12-30T17:50:54Z)
• Neuro-Symbolic Learning of Answer Set Programs from Raw Data [54.56905063752427]
  Neuro-Symbolic AI aims to combine interpretability of symbolic techniques with the ability of deep learning to learn from raw data. We introduce Neuro-Symbolic Inductive Learner (NSIL), an approach that trains a general neural network to extract latent concepts from raw data. NSIL learns expressive knowledge, solves computationally complex problems, and achieves state-of-the-art performance in terms of accuracy and data efficiency.
  arXiv  Detail & Related papers  (2022-05-25T12:41:59Z)
• Closed Loop Neural-Symbolic Learning via Integrating Neural Perception, Grammar Parsing, and Symbolic Reasoning [134.77207192945053]
  Prior methods learn the neural-symbolic models using reinforcement learning approaches. We introduce the textbfgrammar model as a textitsymbolic prior to bridge neural perception and symbolic reasoning. We propose a novel textbfback-search algorithm which mimics the top-down human-like learning procedure to propagate the error.
  arXiv  Detail & Related papers  (2020-06-11T17:42:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.