Spectral Generative Flow Models: A Physics-Inspired Replacement for Vectorized Large Language Models

• URL: http://arxiv.org/abs/2601.08893v2
• Date: Wed, 21 Jan 2026 22:24:36 GMT
• Title: Spectral Generative Flow Models: A Physics-Inspired Replacement for Vectorized Large Language Models
• Authors: Andrew Kiruluta,
• Abstract summary: We introduce Spectral Generative Flow Models (SGFMs), a physics-inspired alternative to transformer-based large language models.<n>Instead of representing text or video as sequences of discrete tokens processed by attention, SGFMs treat generation as the evolution of a continuous field governed by constrained dynamics.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce Spectral Generative Flow Models (SGFMs), a physics-inspired alternative to transformer-based large language models. Instead of representing text or video as sequences of discrete tokens processed by attention, SGFMs treat generation as the evolution of a continuous field governed by constrained stochastic dynamics in a multiscale wavelet basis. This formulation replaces global attention with local operators, spectral projections, and Navier--Stokes-like transport, yielding a generative mechanism grounded in continuity, geometry, and physical structure. Our framework provides three key innovations: (i) a field-theoretic ontology in which text and video are unified as trajectories of a stochastic partial differential equation; (ii) a wavelet-domain representation that induces sparsity, scale separation, and computational efficiency; and (iii) a constrained stochastic flow that enforces stability, coherence, and uncertainty propagation. Together, these components define a generative architecture that departs fundamentally from autoregressive modeling and diffusion-based approaches. SGFMs offer a principled path toward long-range coherence, multimodal generality, and physically structured inductive bias in next-generation generative models.

Related papers

• Bridging the Discrete-Continuous Gap: Unified Multimodal Generation via Coupled Manifold Discrete Absorbing Diffusion [60.186310080523135]
  Bifurcation of generative modeling into autoregressive approaches for discrete data (text) and diffusion approaches for continuous data (images) hinders development of truly unified multimodal systems.<n>We propose textbfCoM-DAD, a novel probabilistic framework that reformulates multimodal generation as a hierarchical dual-process.<n>Our method demonstrates superior stability over standard masked modeling, establishing a new paradigm for scalable, unified text-image generation.
  arXiv  Detail & Related papers  (2026-01-07T16:21:19Z)
• Foundations of Diffusion Models in General State Spaces: A Self-Contained Introduction [54.95522167029998]
  This article is a self-contained primer on diffusion over general state spaces.<n>We develop the discrete-time view (forward noising via Markov kernels and learned reverse dynamics) alongside its continuous-time limits.<n>A common variational treatment yields the ELBO that underpins standard training losses.
  arXiv  Detail & Related papers  (2025-12-04T18:55:36Z)
• Kuramoto Orientation Diffusion Models [67.0711709825854]
  Orientation-rich images, such as fingerprints and textures, often exhibit coherent angular patterns.<n>Motivated by the role of phase synchronization in biological systems, we propose a score-based generative model.<n>We implement competitive results on general image benchmarks and significantly improves generation quality on orientation-dense datasets like fingerprints and textures.
  arXiv  Detail & Related papers  (2025-09-18T18:18:49Z)
• The Information Dynamics of Generative Diffusion [12.52425103385255]
  Generative diffusion models have emerged as a powerful class of models in machine learning.<n>This paper provides an integrated perspective on generative diffusion by connecting their dynamic, information-theoretic, and thermodynamic properties.
  arXiv  Detail & Related papers  (2025-08-27T13:53:56Z)
• Empirical Investigation of Latent Representational Dynamics in Large Language Models: A Manifold Evolution Perspective [4.935224714809964]
  This paper introduces the Dynamical Manifold Evolution Theory (DMET), a conceptual framework that models large language model (LLM) generation as a continuous trajectory evolving on a low-dimensional semantic manifold.<n>The theory characterizes latent dynamics through three interpretable metrics-state continuity ($C$), attractor compactness ($Q$), and topological persistence ($P$)-which jointly capture the smoothness, stability, and structure of representation evolution.
  arXiv  Detail & Related papers  (2025-05-24T14:17:50Z)
• A Unified Approach to Analysis and Design of Denoising Markov Models [11.975300242253496]
  We aim to establish a rigorous mathematical foundation for denoising Markov models.<n>We propose a minimal set of assumptions that ensure explicit construction of the backward generator.<n>Our framework unifies existing formulations of continuous and discrete diffusion models.
  arXiv  Detail & Related papers  (2025-04-02T17:46:43Z)
• Geometric Trajectory Diffusion Models [58.853975433383326]
  Generative models have shown great promise in generating 3D geometric systems. Existing approaches only operate on static structures, neglecting the fact that physical systems are always dynamic in nature. We propose geometric trajectory diffusion models (GeoTDM), the first diffusion model for modeling the temporal distribution of 3D geometric trajectories.
  arXiv  Detail & Related papers  (2024-10-16T20:36:41Z)
• Transformers from Diffusion: A Unified Framework for Neural Message Passing [79.9193447649011]
  Message passing neural networks (MPNNs) have become a de facto class of model solutions.<n>We propose an energy-constrained diffusion model, which integrates the inductive bias of diffusion with layer-wise constraints of energy.<n>Building on these insights, we devise a new class of message passing models, dubbed Transformers (DIFFormer), whose global attention layers are derived from the principled energy-constrained diffusion framework.
  arXiv  Detail & Related papers  (2024-09-13T17:54:41Z)
• Latent Space Energy-based Neural ODEs [73.01344439786524]
  This paper introduces novel deep dynamical models designed to represent continuous-time sequences.<n>We train the model using maximum likelihood estimation with Markov chain Monte Carlo.<n> Experimental results on oscillating systems, videos and real-world state sequences (MuJoCo) demonstrate that our model with the learnable energy-based prior outperforms existing counterparts.
  arXiv  Detail & Related papers  (2024-09-05T18:14:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.