On the Solvability of the {XOR} Problem by Spiking Neural Networks

• URL: http://arxiv.org/abs/2408.05845v1
• Date: Sun, 11 Aug 2024 19:01:39 GMT
• Title: On the Solvability of the {XOR} Problem by Spiking Neural Networks
• Abstract summary: The linearly inseparable XOR problem and the related problem of representing binary logical gates is revisited. We use this problem as an example to study the effect of different hyper parameters such as information encoding. We show that it is possible to realize sparse solutions of a minimal configuration with only two neurons in the hidden layer.
• Score: 0.46040036610482665
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The linearly inseparable XOR problem and the related problem of representing binary logical gates is revisited from the point of view of temporal encoding and its solvability by spiking neural networks with minimal configurations of leaky integrate-and-fire (LIF) neurons. We use this problem as an example to study the effect of different hyper parameters such as information encoding, the number of hidden units in a fully connected reservoir, the choice of the leaky parameter and the reset mechanism in terms of reset-to-zero and reset-by-subtraction based on different refractory times. The distributions of the weight matrices give insight into the difficulty, respectively the probability, to find a solution. This leads to the observation that zero refractory time together with graded spikes and an adapted reset mechanism, reset-to-mod, makes it possible to realize sparse solutions of a minimal configuration with only two neurons in the hidden layer to resolve all binary logic gate constellations with XOR as a special case.

Related papers

• Regularized Large Neighborhood Search [11.54620327711825]
  Operations research practitioners typically tackle NP-hard problems using large neighborhood search (LNS)<n>We bridge this gap by regularized LNS (RLNS)<n>Undertropic regularization, we prove that RLNS performs exact block Gibbs sampling.
  arXiv  Detail & Related papers  (2026-06-01T14:16:18Z)
• Solving Inverse Problems via Diffusion Optimal Control [3.0079490585515343]
  We derive a diffusion-based optimal controller inspired by the iterative Linear Quadratic Regulator (iLQR) algorithm. We show that the idealized posterior sampling equation can be recovered as a special case of our algorithm. We then evaluate our method against a selection of neural inverse problem solvers, and establish a new baseline in image reconstruction with inverse problems.
  arXiv  Detail & Related papers  (2024-12-21T19:47:06Z)
• Score-based Neural Ordinary Differential Equations for Computing Mean Field Control Problems [13.285775352653546]
  This paper proposes a system of neural differential equations representing first- and second-order score functions along trajectories based on deep neural networks. We reformulate the mean viscous field control (MFC) problem with individual noises into an unconstrained optimization problem framed by the proposed neural ODE system.
  arXiv  Detail & Related papers  (2024-09-24T21:45:55Z)
• LinSATNet: The Positive Linear Satisfiability Neural Networks [116.65291739666303]
  This paper studies how to introduce the popular positive linear satisfiability to neural networks. We propose the first differentiable satisfiability layer based on an extension of the classic Sinkhorn algorithm for jointly encoding multiple sets of marginal distributions.
  arXiv  Detail & Related papers  (2024-07-18T22:05:21Z)
• Q2A: Querying Implicit Fully Continuous Feature Pyramid to Align Features for Medical Image Segmentation [10.66113563605316]
  We propose Q2A, a novel one-step query-based aligning paradigm, to solve the feature misalignment problem in the INR-based decoder. For each target coordinate, Q2A first generates several queries depicting the spatial offsets and the cell resolutions of the contextual features aligned to the coordinate, then calculates the corresponding aligned features.
  arXiv  Detail & Related papers  (2024-04-15T05:53:26Z)
• Adaptive Multilevel Neural Networks for Parametric PDEs with Error Estimation [0.0]
  A neural network architecture is presented to solve high-dimensional parameter-dependent partial differential equations (pPDEs) It is constructed to map parameters of the model data to corresponding finite element solutions. It outputs a coarse grid solution and a series of corrections as produced in an adaptive finite element method (AFEM)
  arXiv  Detail & Related papers  (2024-03-19T11:34:40Z)
• Solving Elliptic Optimal Control Problems via Neural Networks and Optimality System [3.8704302640118864]
  We investigate a neural network based solver for optimal control problems (without / with box constraint) It employs deep neural networks to represent the solutions to the reduced system. We present several numerical examples to illustrate the method and compare it with two existing ones.
  arXiv  Detail & Related papers  (2023-08-23T05:18:19Z)
• On the Effective Number of Linear Regions in Shallow Univariate ReLU Networks: Convergence Guarantees and Implicit Bias [50.84569563188485]
  We show that gradient flow converges in direction when labels are determined by the sign of a target network with $r$ neurons. Our result may already hold for mild over- parameterization, where the width is $tildemathcalO(r)$ and independent of the sample size.
  arXiv  Detail & Related papers  (2022-05-18T16:57:10Z)
• Mean-field Analysis of Piecewise Linear Solutions for Wide ReLU Networks [83.58049517083138]
  We consider a two-layer ReLU network trained via gradient descent. We show that SGD is biased towards a simple solution. We also provide empirical evidence that knots at locations distinct from the data points might occur.
  arXiv  Detail & Related papers  (2021-11-03T15:14:20Z)
• The Separation Capacity of Random Neural Networks [78.25060223808936]
  We show that a sufficiently large two-layer ReLU-network with standard Gaussian weights and uniformly distributed biases can solve this problem with high probability. We quantify the relevant structure of the data in terms of a novel notion of mutual complexity.
  arXiv  Detail & Related papers  (2021-07-31T10:25:26Z)
• LocalDrop: A Hybrid Regularization for Deep Neural Networks [98.30782118441158]
  We propose a new approach for the regularization of neural networks by the local Rademacher complexity called LocalDrop. A new regularization function for both fully-connected networks (FCNs) and convolutional neural networks (CNNs) has been developed based on the proposed upper bound of the local Rademacher complexity.
  arXiv  Detail & Related papers  (2021-03-01T03:10:11Z)
• On Connectivity of Solutions in Deep Learning: The Role of Over-parameterization and Feature Quality [21.13299067136635]
  We present a novel condition for ensuring the connectivity of two arbitrary points in parameter space. This condition is provably milder than dropout stability, and it provides a connection between the problem of finding low-loss paths and the memorization capacity of neural nets.
  arXiv  Detail & Related papers  (2021-02-18T23:44:08Z)
• Joint Deep Reinforcement Learning and Unfolding: Beam Selection and Precoding for mmWave Multiuser MIMO with Lens Arrays [54.43962058166702]
  millimeter wave (mmWave) multiuser multiple-input multiple-output (MU-MIMO) systems with discrete lens arrays have received great attention. In this work, we investigate the joint design of a beam precoding matrix for mmWave MU-MIMO systems with DLA.
  arXiv  Detail & Related papers  (2021-01-05T03:55:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.