Variational Distributional Neuron

• URL: http://arxiv.org/abs/2602.18250v1
• Date: Fri, 20 Feb 2026 14:35:53 GMT
• Title: Variational Distributional Neuron
• Abstract summary: A compute unit formulated as a VAE brick carries a prior, an amortized posterior and a local ELBO.<n>We analyze "collapse" modes and the conditions for a "living neuron"<n>We then extend the contribution over time via autoregressive priors over the latent, per unit.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose a proof of concept for a variational distributional neuron: a compute unit formulated as a VAE brick, explicitly carrying a prior, an amortized posterior and a local ELBO. The unit is no longer a deterministic scalar but a distribution: computing is no longer about propagating values, but about contracting a continuous space of possibilities under constraints. Each neuron parameterizes a posterior, propagates a reparameterized sample and is regularized by the KL term of a local ELBO - hence, the activation is distributional. This "contraction" becomes testable through local constraints and can be monitored via internal measures. The amount of contextual information carried by the unit, as well as the temporal persistence of this information, are locally tuned by distinct constraints. This proposal addresses a structural tension: in sequential generation, causality is predominantly organized in the symbolic space and, even when latents exist, they often remain auxiliary, while the effective dynamics are carried by a largely deterministic decoder. In parallel, probabilistic latent models capture factors of variation and uncertainty, but that uncertainty typically remains borne by global or parametric mechanisms, while units continue to propagate scalars - hence the pivot question: if uncertainty is intrinsic to computation, why does the compute unit not carry it explicitly? We therefore draw two axes: (i) the composition of probabilistic constraints, which must be made stable, interpretable and controllable; and (ii) granularity: if inference is a negotiation of distributions under constraints, should the primitive unit remain deterministic or become distributional? We analyze "collapse" modes and the conditions for a "living neuron", then extend the contribution over time via autoregressive priors over the latent, per unit.

Related papers

• A Compositional Theory of Causally Masked Transformers [50.88598486616582]
  We develop a formalization that derives expressivity directly from the model's implemented dynamics.<n>Each attention head updates its own state independently within a layer.<n>A modified form of soft attention supports irreversible, checklist-like state.
  arXiv  Detail & Related papers  (2026-07-29T14:47:19Z)
• The conditional-mean barrier: From deterministic regression to conditional distribution learning [6.160113667162026]
  This tutorial develops a self-contained module centered on the conditional-mean barrier.<n>Adding latent randomness to a squared-loss predictor collapses it back to the conditional mean.<n>We briefly organize common distributional objectives, including negative log-likelihood, moment and observable matching, variational objectives, adversarial divergences, and score matching.
  arXiv  Detail & Related papers  (2026-05-27T07:31:42Z)
• Why SGD is not Brownian Motion: A New Perspective on Stochastic Dynamics [92.39053980710702]
  Gradient Descent (SGD) is commonly modeled as a Langevin process, assuming that minibatch noise acts as Brownian motion.<n>This approximation relies on a continuous-time limit and a sqrt(eta) noise scaling that does not match the discrete SGD update at finite learning rate.<n>We propose an alternative formulation of SGD as deterministic dynamics in a fluctuating loss landscape induced by minibatch sampling.
  arXiv  Detail & Related papers  (2026-05-21T15:50:40Z)
• Randomized Atomic Feature Models for Physics-Informed Identification of Dynamic Systems [3.810612452609132]
  We present a framework for system identification based on randomized stable atomic features.<n>The framework encodes stability margins, modal localization, DC-gain bounds, monotonicity, passivity, relative degree, settling-time targets, and time/frequency-domain error bounds.
  arXiv  Detail & Related papers  (2026-05-14T04:25:07Z)
• Exploring the Dimensions of a Variational Neuron [0.0]
  We introduce EVE, a variational distributional neuron formulated as a local probabilistic computational unit.<n>Eve is instrumented with internal diagnostics and constraints, including effective KL, a target band on mu2, out-of-band fractions, and indicators of drift and collapse.<n>Overall, the paper provides an experimentally grounded first map of the design space opened by a variational neuron.
  arXiv  Detail & Related papers  (2026-03-14T09:09:13Z)
• Deep Sequence Modeling with Quantum Dynamics: Language as a Wave Function [0.0]
  We introduce a sequence modeling framework in which the latent state is a complex-valued wave function evolving on a finite-dimensional Hilbert space under a learned, time-dependent Hamiltonian.<n> Token probabilities are extracted using the Born rule, a quadratic measurement operator that couples magnitudes and relative phases.<n>We derive a continuity equation for the latent probability mass, yielding conserved pairwise currents that serve as a built-in diagnostic.
  arXiv  Detail & Related papers  (2026-02-24T23:42:18Z)
• Random-Matrix-Induced Simplicity Bias in Over-parameterized Variational Quantum Circuits [72.0643009153473]
  We show that expressive variational ansatze enter a Haar-like universality class in which both observable expectation values and parameter gradients concentrate exponentially with system size.<n>As a consequence, the hypothesis class induced by such circuits collapses with high probability to a narrow family of near-constant functions.<n>We further show that this collapse is not unavoidable: tensor-structured VQCs, including tensor-network-based and tensor-hypernetwork parameterizations, lie outside the Haar-like universality class.
  arXiv  Detail & Related papers  (2026-01-05T08:04:33Z)
• Chaos, Entanglement and Measurement: Field-Theoretic Perspectives on Quantum Information Dynamics [0.0]
  I study scrambling and pseudorandomness in the Brownian Sachdev-Ye-Kitaev (SYK) model.<n>I construct a field theory for weakly measured SYK clusters.<n>I develop a strong-disorder renormalization group for measurement-only SYK clusters.
  arXiv  Detail & Related papers  (2025-12-11T10:04:30Z)
• On the entanglement of co-ordinate and momentum degrees of freedom in noncommutative space [0.0]
  We investigate the quantum entanglement induced by phase-space noncommutativity. The entanglement properties of coordinate and momentum degrees of freedom are studied. We show that the mere inclusion of non-commutativity of phase-space is not sufficient to generate the entanglement.
  arXiv  Detail & Related papers  (2024-01-05T18:43:47Z)
• Diagnosing non-Hermitian Many-Body Localization and Quantum Chaos via Singular Value Decomposition [0.0]
  Strong local disorder in interacting quantum spin chains can turn delocalized eigenmodes into localized eigenstates. This is accompanied by distinct spectral statistics: chaotic for the delocalized phase and integrable for the localized phase. We ask whether random dissipation (without random disorder) can induce chaotic or localized behavior in an otherwise integrable system.
  arXiv  Detail & Related papers  (2023-11-27T19:00:01Z)
• Machine learning in and out of equilibrium [58.88325379746631]
  Our study uses a Fokker-Planck approach, adapted from statistical physics, to explore these parallels. We focus in particular on the stationary state of the system in the long-time limit, which in conventional SGD is out of equilibrium. We propose a new variation of Langevin dynamics (SGLD) that harnesses without replacement minibatching.
  arXiv  Detail & Related papers  (2023-06-06T09:12:49Z)
• A probabilistic autoencoder for causal discovery [0.0]
  The paper addresses the problem of finding the causal direction between two associated variables. The proposed solution is to build an autoencoder of their joint distribution and to maximize its estimation capacity.
  arXiv  Detail & Related papers  (2022-12-08T12:33:16Z)
• Nonparametric Conditional Local Independence Testing [69.31200003384122]
  Conditional local independence is an independence relation among continuous time processes. No nonparametric test of conditional local independence has been available. We propose such a nonparametric test based on double machine learning.
  arXiv  Detail & Related papers  (2022-03-25T10:31:02Z)
• Neuro-Symbolic Entropy Regularization [78.16196949641079]
  In structured prediction, the goal is to jointly predict many output variables that together encode a structured object. One approach -- entropy regularization -- posits that decision boundaries should lie in low-probability regions. We propose a loss, neuro-symbolic entropy regularization, that encourages the model to confidently predict a valid object.
  arXiv  Detail & Related papers  (2022-01-25T06:23:10Z)
• Discovering Latent Causal Variables via Mechanism Sparsity: A New Principle for Nonlinear ICA [81.4991350761909]
  Independent component analysis (ICA) refers to an ensemble of methods which formalize this goal and provide estimation procedure for practical application. We show that the latent variables can be recovered up to a permutation if one regularizes the latent mechanisms to be sparse.
  arXiv  Detail & Related papers  (2021-07-21T14:22:14Z)
• Interplay between transport and quantum coherences in free fermionic systems [58.720142291102135]
  We study the quench dynamics in free fermionic systems. In particular, we identify a function, that we dub emphtransition map, which takes the value of the stationary current as input and gives the value of correlation as output.
  arXiv  Detail & Related papers  (2021-03-24T17:47:53Z)
• Unifying supervised learning and VAEs -- coverage, systematics and goodness-of-fit in normalizing-flow based neural network models for astro-particle reconstructions [0.0]
  Statistical uncertainties, coverage, systematic uncertainties or a goodness-of-fit measure are often not calculated. We show that a KL-divergence objective of the joint distribution of data and labels allows to unify supervised learning and variational autoencoders. We discuss how to calculate coverage probabilities without numerical integration for specific "base-ordered" contours.
  arXiv  Detail & Related papers  (2020-08-13T11:28:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.