Separable neural architectures as a primitive for unified predictive and generative intelligence

• URL: http://arxiv.org/abs/2603.12244v2
• Date: Sun, 15 Mar 2026 18:39:13 GMT
• Title: Separable neural architectures as a primitive for unified predictive and generative intelligence
• Abstract summary: We formalise a representational class that unifies additive, quadratic and tensor-decomposed neural models.<n>SNAs impose a structural inductive bias that factorises high-dimensional mappings into low-composed components.<n>These results establish the separable neural architecture as the domain-agnostic primitive for predictive and generative intelligence.
• Score: 0.7318787399845128
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Intelligent systems across physics, language and perception often exhibit factorisable structure, yet are typically modelled by monolithic neural architectures that do not explicitly exploit this structure. The separable neural architecture (SNA) addresses this by formalising a representational class that unifies additive, quadratic and tensor-decomposed neural models. By constraining interaction order and tensor rank, SNAs impose a structural inductive bias that factorises high-dimensional mappings into low-arity components. Separability need not be a property of the system itself: it often emerges in the coordinates or representations through which the system is expressed. Crucially, this coordinate-aware formulation reveals a structural analogy between chaotic spatiotemporal dynamics and linguistic autoregression. By treating continuous physical states as smooth, separable embeddings, SNAs enable distributional modelling of chaotic systems. This approach mitigates the nonphysical drift characteristics of deterministic operators whilst remaining applicable to discrete sequences. The compositional versatility of this approach is demonstrated across four domains: autonomous waypoint navigation via reinforcement learning, inverse generation of multifunctional microstructures, distributional modelling of turbulent flow and neural language modelling. These results establish the separable neural architecture as a domain-agnostic primitive for predictive and generative intelligence, capable of unifying both deterministic and distributional representations.

Related papers

• Neural operator discovery from heterogeneous trajectories [3.174604660846279]
  We introduce a factorized latent-conditioning formulation that jointly learns a neural operator and a low-dimensional latent representation.<n>Across diverse systems, the learned latent representation captures the intrinsic dimensionality of system variation.<n>Results establish an interpretable paradigm for operator learning in the absence of explicit factor supervision.
  arXiv  Detail & Related papers  (2026-07-25T19:20:25Z)
• Toward a mechanistic understanding of inference in visual cortex and diffusion models [46.86106767256413]
  We describe a model of perceptual inference in primary visual cortex (V1)<n>We efficiently train these recurrent dynamics using a denoising score-matching objective and implicit differentiation.<n>This model exhibits exceptionally good denoising performance, restoring image features such as extended contours amid extreme visual ambiguity.
  arXiv  Detail & Related papers  (2026-07-17T07:11:57Z)
• Topological Neural Dynamics: A Neuron-wise Framework for Sequence Modeling [55.07411490538404]
  Topological Neural Dynamics (TND) is a sequence modeling framework that shifts from layer-wise to neuron-wise dynamics.<n>TND represents a neural system as a directed neuron graph, an interaction operator, and a local dynamics function.<n>We evaluate TND as a case study on a behavior cloning task in single-player Pong.
  arXiv  Detail & Related papers  (2026-06-19T10:18:24Z)
• Kirchhoff-Inspired Neural Networks for Evolving High-Order Perception [51.28469959548178]
  We propose the Kirchhoff-Inspired Neural Network (KINN), a state-variable-based network architecture constructed based on Kirchhoff's current law.<n>KINN derives numerically stable state updates from fundamental ordinary differential equations, enabling the explicit decoupling and encoding of higher-order evolutionary components.
  arXiv  Detail & Related papers  (2026-03-25T06:17:31Z)
• Deep Neural Networks as Iterated Function Systems and a Generalization Bound [2.7920304852537536]
  We show that two important deep architectures can be viewed as, or canonically associated with, place-dependent IFS.<n>We derive a Wasserstein bound for generative modeling that controls the collage-type approximation error between the data distribution and its image.
  arXiv  Detail & Related papers  (2026-01-27T07:32:49Z)
• Scaling Laws and Representation Learning in Simple Hierarchical Languages: Transformers vs. Convolutional Architectures [49.19753720526998]
  We derive theoretical scaling laws for neural network performance on synthetic datasets.<n>We validate that convolutional networks, whose structure aligns with that of the generative process through locality and weight sharing, enjoy a faster scaling of performance.<n>This finding clarifies the architectural biases underlying neural scaling laws and highlights how representation learning is shaped by the interaction between model architecture and the statistical properties of data.
  arXiv  Detail & Related papers  (2025-05-11T17:44:14Z)
• Discrete, compositional, and symbolic representations through attractor dynamics [51.20712945239422]
  We introduce a novel neural systems model that integrates attractor dynamics with symbolic representations to model cognitive processes akin to the probabilistic language of thought (PLoT) Our model segments the continuous representational space into discrete basins, with attractor states corresponding to symbolic sequences, that reflect the semanticity and compositionality characteristic of symbolic systems through unsupervised learning, rather than relying on pre-defined primitives. This approach establishes a unified framework that integrates both symbolic and sub-symbolic processing through neural dynamics, a neuroplausible substrate with proven expressivity in AI, offering a more comprehensive model that mirrors the complex duality of cognitive operations
  arXiv  Detail & Related papers  (2023-10-03T05:40:56Z)
• A Constructive Approach to Function Realization by Neural Stochastic Differential Equations [8.04975023021212]
  We introduce structural restrictions on system dynamics and characterize the class of functions that can be realized by such a system. The systems are implemented as a cascade interconnection of a neural differential equation (Neural SDE), a deterministic dynamical system, and a readout map.
  arXiv  Detail & Related papers  (2023-07-01T03:44:46Z)
• Stretched and measured neural predictions of complex network dynamics [2.1024950052120417]
  Data-driven approximations of differential equations present a promising alternative to traditional methods for uncovering a model of dynamical systems. A recently employed machine learning tool for studying dynamics is neural networks, which can be used for data-driven solution finding or discovery of differential equations. We show that extending the model's generalizability beyond traditional statistical learning theory limits is feasible.
  arXiv  Detail & Related papers  (2023-01-12T09:44:59Z)
• Neural-Symbolic Recursive Machine for Systematic Generalization [113.22455566135757]
  We introduce the Neural-Symbolic Recursive Machine (NSR), whose core is a Grounded Symbol System (GSS) NSR integrates neural perception, syntactic parsing, and semantic reasoning. We evaluate NSR's efficacy across four challenging benchmarks designed to probe systematic generalization capabilities.
  arXiv  Detail & Related papers  (2022-10-04T13:27:38Z)
• Universal approximation property of invertible neural networks [76.95927093274392]
  Invertible neural networks (INNs) are neural network architectures with invertibility by design. Thanks to their invertibility and the tractability of Jacobian, INNs have various machine learning applications such as probabilistic modeling, generative modeling, and representation learning.
  arXiv  Detail & Related papers  (2022-04-15T10:45:26Z)
• Modeling Structure with Undirected Neural Networks [20.506232306308977]
  We propose undirected neural networks, a flexible framework for specifying computations that can be performed in any order. We demonstrate the effectiveness of undirected neural architectures, both unstructured and structured, on a range of tasks.
  arXiv  Detail & Related papers  (2022-02-08T10:06:51Z)
• Stability of Algebraic Neural Networks to Small Perturbations [179.55535781816343]
  Algebraic neural networks (AlgNNs) are composed of a cascade of layers each one associated to and algebraic signal model. We show how any architecture that uses a formal notion of convolution can be stable beyond particular choices of the shift operator.
  arXiv  Detail & Related papers  (2020-10-22T09:10:16Z)
• Provably Efficient Neural Estimation of Structural Equation Model: An Adversarial Approach [144.21892195917758]
  We study estimation in a class of generalized Structural equation models (SEMs) We formulate the linear operator equation as a min-max game, where both players are parameterized by neural networks (NNs), and learn the parameters of these neural networks using a gradient descent. For the first time we provide a tractable estimation procedure for SEMs based on NNs with provable convergence and without the need for sample splitting.
  arXiv  Detail & Related papers  (2020-07-02T17:55:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.