KLay: Accelerating Neurosymbolic AI

• URL: http://arxiv.org/abs/2410.11415v1
• Date: Tue, 15 Oct 2024 09:02:55 GMT
• Title: KLay: Accelerating Neurosymbolic AI
• Authors: Jaron Maene, Vincent Derkinderen, Pedro Zuidberg Dos Martires,
• Abstract summary: A popular approach to neurosymbolic AI involves mapping logic formulas to arithmetic circuits. We introduce knowledge layers (KLay) to represent arithmetic circuits that can be efficiently parallelized. We empirically show that KLay achieves speedups of multiple orders of magnitude over the state of the art.
• Score: 5.342751235300229
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A popular approach to neurosymbolic AI involves mapping logic formulas to arithmetic circuits (computation graphs consisting of sums and products) and passing the outputs of a neural network through these circuits. This approach enforces symbolic constraints onto a neural network in a principled and end-to-end differentiable way. Unfortunately, arithmetic circuits are challenging to run on modern AI accelerators as they exhibit a high degree of irregular sparsity. To address this limitation, we introduce knowledge layers (KLay), a new data structure to represent arithmetic circuits that can be efficiently parallelized on GPUs. Moreover, we contribute two algorithms used in the translation of traditional circuit representations to KLay and a further algorithm that exploits parallelization opportunities during circuit evaluations. We empirically show that KLay achieves speedups of multiple orders of magnitude over the state of the art, thereby paving the way towards scaling neurosymbolic AI to larger real-world applications.

Related papers

• NeuralFastLAS: Fast Logic-Based Learning from Raw Data [54.938128496934695]
  Symbolic rule learners generate interpretable solutions, however they require the input to be encoded symbolically. Neuro-symbolic approaches overcome this issue by mapping raw data to latent symbolic concepts using a neural network. We introduce NeuralFastLAS, a scalable and fast end-to-end approach that trains a neural network jointly with a symbolic learner.
  arXiv  Detail & Related papers  (2023-10-08T12:33:42Z)
• CktGNN: Circuit Graph Neural Network for Electronic Design Automation [67.29634073660239]
  This paper presents a Circuit Graph Neural Network (CktGNN) that simultaneously automates the circuit topology generation and device sizing. We introduce Open Circuit Benchmark (OCB), an open-sourced dataset that contains $10$K distinct operational amplifiers. Our work paves the way toward a learning-based open-sourced design automation for analog circuits.
  arXiv  Detail & Related papers  (2023-08-31T02:20:25Z)
• Injecting Logical Constraints into Neural Networks via Straight-Through Estimators [5.6613898352023515]
  Injecting discrete logical constraints into neural network learning is one of the main challenges in neuro-symbolic AI. We find that a straight-through-estimator, a method introduced to train binary neural networks, could effectively be applied to incorporate logical constraints into neural network learning.
  arXiv  Detail & Related papers  (2023-07-10T05:12:05Z)
• Pathfinding Neural Cellular Automata [23.831530224401575]
  Pathfinding is an important sub-component of a broad range of complex AI tasks, such as robot path planning, transport routing, and game playing. We hand-code and learn models for Breadth-First Search (BFS), i.e. shortest path finding. We present a neural implementation of Depth-First Search (DFS), and outline how it can be combined with neural BFS to produce an NCA for computing diameter of a graph. We experiment with architectural modifications inspired by these hand-coded NCAs, training networks from scratch to solve the diameter problem on grid mazes while exhibiting strong ability generalization
  arXiv  Detail & Related papers  (2023-01-17T11:45:51Z)
• Intelligence Processing Units Accelerate Neuromorphic Learning [52.952192990802345]
  Spiking neural networks (SNNs) have achieved orders of magnitude improvement in terms of energy consumption and latency. We present an IPU-optimized release of our custom SNN Python package, snnTorch.
  arXiv  Detail & Related papers  (2022-11-19T15:44:08Z)
• Accelerating the training of single-layer binary neural networks using the HHL quantum algorithm [58.720142291102135]
  We show that useful information can be extracted from the quantum-mechanical implementation of Harrow-Hassidim-Lloyd (HHL) This paper shows, however, that useful information can be extracted from the quantum-mechanical implementation of HHL, and used to reduce the complexity of finding the solution on the classical side.
  arXiv  Detail & Related papers  (2022-10-23T11:58:05Z)
• Pretraining Graph Neural Networks for few-shot Analog Circuit Modeling and Design [68.1682448368636]
  We present a supervised pretraining approach to learn circuit representations that can be adapted to new unseen topologies or unseen prediction tasks. To cope with the variable topological structure of different circuits we describe each circuit as a graph and use graph neural networks (GNNs) to learn node embeddings. We show that pretraining GNNs on prediction of output node voltages can encourage learning representations that can be adapted to new unseen topologies or prediction of new circuit level properties.
  arXiv  Detail & Related papers  (2022-03-29T21:18:47Z)
• Tractable Boolean and Arithmetic Circuits [11.358487655918676]
  We review the foundations of tractable circuits and some associated milestones. We focus on their core properties and techniques that make them particularly useful for the broad aims of neuro-symbolic AI.
  arXiv  Detail & Related papers  (2022-02-07T05:01:38Z)
• Monolithic Silicon Photonic Architecture for Training Deep Neural Networks with Direct Feedback Alignment [0.6501025489527172]
  We propose on-chip training of neural networks enabled by a CMOS-compatible silicon photonic architecture. Our scheme employs the direct feedback alignment training algorithm, which trains neural networks using error feedback rather than error backpropagation. We experimentally demonstrate training a deep neural network with the MNIST dataset using on-chip MAC operation results.
  arXiv  Detail & Related papers  (2021-11-12T18:31:51Z)
• On-Chip Error-triggered Learning of Multi-layer Memristive Spiking Neural Networks [1.7958576850695402]
  We propose a local, gradient-based, error-triggered learning algorithm with online ternary weight updates. The proposed algorithm enables online training of multi-layer SNNs with memristive neuromorphic hardware.
  arXiv  Detail & Related papers  (2020-11-21T19:44:19Z)
• Training End-to-End Analog Neural Networks with Equilibrium Propagation [64.0476282000118]
  We introduce a principled method to train end-to-end analog neural networks by gradient descent. We show mathematically that a class of analog neural networks (called nonlinear resistive networks) are energy-based models. Our work can guide the development of a new generation of ultra-fast, compact and low-power neural networks supporting on-chip learning.
  arXiv  Detail & Related papers  (2020-06-02T23:38:35Z)
• One-step regression and classification with crosspoint resistive memory arrays [62.997667081978825]
  High speed, low energy computing machines are in demand to enable real-time artificial intelligence at the edge. One-step learning is supported by simulations of the prediction of the cost of a house in Boston and the training of a 2-layer neural network for MNIST digit recognition. Results are all obtained in one computational step, thanks to the physical, parallel, and analog computing within the crosspoint array.
  arXiv  Detail & Related papers  (2020-05-05T08:00:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.