On the Sampling Sparsity of Neuromorphic Analog-to-Spike Conversion based on Leaky Integrate-and-Fire

• URL: http://arxiv.org/abs/2410.17441v1
• Date: Tue, 22 Oct 2024 21:42:07 GMT
• Title: On the Sampling Sparsity of Neuromorphic Analog-to-Spike Conversion based on Leaky Integrate-and-Fire
• Authors: Bernhard A. Moser, Michael Lunglmayr,
• Abstract summary: We show in a rigorous mathematical way that information encoding by means of Threshold-Based Representation is linked to an analog-to-spike conversion. In contrast to the traditional principle of periodic sensing neuromorphic engineering pursues a paradigm shift towards bio-inspired event-based sensing.
• Score: 0.46040036610482665
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In contrast to the traditional principle of periodic sensing neuromorphic engineering pursues a paradigm shift towards bio-inspired event-based sensing, where events are primarily triggered by a change in the perceived stimulus. We show in a rigorous mathematical way that information encoding by means of Threshold-Based Representation based on either Leaky Integrate-and-Fire (LIF) or Send-on-Delta (SOD) is linked to an analog-to-spike conversion that guarantees maximum sparsity while satisfying an approximation condition based on the Alexiewicz norm.

Related papers

• Optimal Effective Hamiltonian for Quantum Computing and Simulation [1.0359978670015826]
  We establish the Least Action Unitary Transformation as the fundamental principle for effective models.<n>We validate this framework against experimental data from superconducting quantum processors.<n>This work provides a systematic, experimentally validated route for high-precision system learning and Hamiltonian engineering.
  arXiv  Detail & Related papers  (2026-02-03T15:09:29Z)
• Joint Source-Channel-Generation Coding: From Distortion-oriented Reconstruction to Semantic-consistent Generation [58.67925548779465]
  We propose Joint Source-Channel-Generation Coding (JSCGC), a novel paradigm that shifts the focus from perceptual reconstruction to probabilistic generation.<n>JSCGC improves substantially semantic quality and semantic fidelity, significantly outperforming conventional distortion-oriented J SCC methods.
  arXiv  Detail & Related papers  (2026-01-19T08:12:47Z)
• Activation Sensitivity as a Unifying Principle for Post-Training Quantization [0.0]
  Post-training quantization (PTQ) methods for large language models rely on prioritizing which weight channels most strongly influence model behavior.<n>We present a unified theoretical framework for PTQ by formalizing activation sensitivity, defined as the expected impact of channel-wise perturbations on the loss.<n>We analyze the design space of sensitivity metrics, connect gradient-based saliency, Fisher information, and Hessian-based criteria, and clarify their relationships to classical pruning methods such as Optimal Brain Damage and Optimal Brain Surgeon.
  arXiv  Detail & Related papers  (2026-01-15T22:27:19Z)
• Beyond Confidence: Adaptive and Coherent Decoding for Diffusion Language Models [64.92045568376705]
  Coherent Contextual Decoding (CCD) is a novel inference framework built upon two core innovations.<n>CCD employs a trajectory rectification mechanism that leverages historical context to enhance sequence coherence.<n>Instead of rigid allocations based on diffusion steps, we introduce an adaptive sampling strategy that dynamically adjusts the unmasking budget for each step.
  arXiv  Detail & Related papers  (2025-11-26T09:49:48Z)
• Flow Equivariant Recurrent Neural Networks [2.900810893770134]
  In machine learning, neural network architectures that respect symmetries of their data are called equivariant.<n>We extend equivariant network theory to this regime of flows', capturing natural transformations over time.<n>We show that these models significantly outperform their non-equivariant counterparts in terms of training speed, length generalization, and velocity generalization.
  arXiv  Detail & Related papers  (2025-07-20T02:52:21Z)
• Integrate-and-Fire from a Mathematical and Signal Processing Perspective [0.40964539027092917]
  Integrate-and-Fire (IF) is an idealized model of the spike-triggering mechanism of a biological neuron. We show that IF is closely related to the concept of Send-on-Delta (SOD) as used in threshold-based sampling.
  arXiv  Detail & Related papers  (2025-01-20T12:39:12Z)
• Learning and Transferring Sparse Contextual Bigrams with Linear Transformers [47.37256334633102]
  We introduce the Sparse Con Bigram model, where the next token's generation depends on a sparse set of earlier positions determined by the last token. We analyze the training dynamics and sample complexity of learning SCB using a one-layer linear transformer with a gradient-based algorithm. We prove that, provided a nontrivial correlation between the downstream and pretraining tasks, finetuning from a pretrained model allows us to bypass the initial sample-intensive stage.
  arXiv  Detail & Related papers  (2024-10-30T20:29:10Z)
• PseudoNeg-MAE: Self-Supervised Point Cloud Learning using Conditional Pseudo-Negative Embeddings [55.55445978692678]
  PseudoNeg-MAE enhances global feature representation of point cloud masked autoencoders by making them both discriminative and sensitive to transformations.<n>We propose a novel loss that explicitly penalizes invariant collapse, enabling the network to capture richer transformation cues while preserving discriminative representations.
  arXiv  Detail & Related papers  (2024-09-24T07:57:21Z)
• A Sampling Theory Perspective on Activations for Implicit Neural Representations [73.6637608397055]
  Implicit Neural Representations (INRs) have gained popularity for encoding signals as compact, differentiable entities. We conduct a comprehensive analysis of these activations from a sampling theory perspective. Our investigation reveals that sinc activations, previously unused in conjunction with INRs, are theoretically optimal for signal encoding.
  arXiv  Detail & Related papers  (2024-02-08T05:52:45Z)
• Machine Learning for the identification of phase-transitions in interacting agent-based systems: a Desai-Zwanzig example [0.0]
  We propose a data-driven framework that pinpoints phase transitions for an agent-based model in its mean-field limit. To this end, we use the manifold learning algorithm Maps to identify a parsimonious set of data-driven latent variables. We then utilize a deep learning framework to obtain a conformal reparametrization of the data-driven coordinates.
  arXiv  Detail & Related papers  (2023-10-29T15:07:08Z)
• Transport meets Variational Inference: Controlled Monte Carlo Diffusions [5.5654189024307685]
  We present a principled and systematic framework for sampling and generative modelling centred around divergences on path space. Our work culminates in the development of the emphControlled Monte Carlo Diffusion sampler (CMCD) for Bayesian computation.
  arXiv  Detail & Related papers  (2023-07-03T14:28:36Z)
• Enhanced Distribution Modelling via Augmented Architectures For Neural ODE Flows [5.33024001730262]
  We present AFFJORD, a neural ODE-based normalizing flow which enhances the representation power of FFJORD. We derive the Jacobian determinant of the general augmented form by generalizing the chain rule in the continuous sense into the Cable Rule. Our experimental results on density estimation in synthetic and high dimensional data, such as MNIST, CIFAR-10 and CelebA 32x32, show that AFFJORD outperforms the baseline FFJORD.
  arXiv  Detail & Related papers  (2023-06-05T09:27:28Z)
• Joint Sensing, Communication, and AI: A Trifecta for Resilient THz User Experiences [118.91584633024907]
  A novel joint sensing, communication, and artificial intelligence (AI) framework is proposed so as to optimize extended reality (XR) experiences over terahertz (THz) wireless systems.
  arXiv  Detail & Related papers  (2023-04-29T00:39:50Z)
• Toward Certified Robustness Against Real-World Distribution Shifts [65.66374339500025]
  We train a generative model to learn perturbations from data and define specifications with respect to the output of the learned model. A unique challenge arising from this setting is that existing verifiers cannot tightly approximate sigmoid activations. We propose a general meta-algorithm for handling sigmoid activations which leverages classical notions of counter-example-guided abstraction refinement.
  arXiv  Detail & Related papers  (2022-06-08T04:09:13Z)
• Invariant Priors for Bayesian Quadrature [8.185918509343818]
  We explore priors that encode invariance of the integrand under a set of transformations in the input domain. We show initial results on superior performance in comparison to standard Bayesian quadrature on several synthetic and one real world application.
  arXiv  Detail & Related papers  (2021-12-02T19:42:04Z)
• Towards Principled Disentanglement for Domain Generalization [90.9891372499545]
  A fundamental challenge for machine learning models is generalizing to out-of-distribution (OOD) data. We first formalize the OOD generalization problem as constrained optimization, called Disentanglement-constrained Domain Generalization (DDG) Based on the transformation, we propose a primal-dual algorithm for joint representation disentanglement and domain generalization.
  arXiv  Detail & Related papers  (2021-11-27T07:36:32Z)
• Stochastic embeddings of dynamical phenomena through variational autoencoders [1.7205106391379026]
  We use a recognition network to increase the observed space dimensionality during the reconstruction of the phase space. Our validation shows that this approach not only recovers a state space that resembles the original one, but it is also able to synthetize new time series.
  arXiv  Detail & Related papers  (2020-10-13T10:10:24Z)
• An Ode to an ODE [78.97367880223254]
  We present a new paradigm for Neural ODE algorithms, called ODEtoODE, where time-dependent parameters of the main flow evolve according to a matrix flow on the group O(d) This nested system of two flows provides stability and effectiveness of training and provably solves the gradient vanishing-explosion problem.
  arXiv  Detail & Related papers  (2020-06-19T22:05:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.