Beyond Attention: True Adaptive World Models via Spherical Kernel Operator

• URL: http://arxiv.org/abs/2603.13263v1
• Date: Wed, 25 Feb 2026 21:51:34 GMT
• Title: Beyond Attention: True Adaptive World Models via Spherical Kernel Operator
• Abstract summary: We introduce Spherical Kernel, a framework that replaces standard attention.<n>SKO mathematically decouples the true environmental transition dynamics from the biased observation frequency of the agent.<n> Empirical evaluations confirm that SKO significantly accelerates convergence and outperforms standard attention baselines in autoregressive language modeling.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The pursuit of world model based artificial intelligence has predominantly relied on projecting high-dimensional observations into parameterized latent spaces, wherein transition dynamics are subsequently learned. However, this conventional paradigm is mathematically flawed: it merely displaces the manifold learning problem into the latent space. When the underlying data distribution shifts, the latent manifold shifts accordingly, forcing the predictive operator to implicitly relearn the new topological structure. Furthermore, by classical approximation theory, positive operators like dot product attention inevitably suffer from the saturation phenomenon, permanently bottlenecking their predictive capacity and leaving them vulnerable to the curse of dimensionality. In this paper, we formulate a mathematically rigorous paradigm for world model construction by redefining the core predictive mechanism. Inspired by Ryan O'Dowd's foundational work we introduce Spherical Kernel Operator (SKO), a framework that replaces standard attention. By projecting the unknown data manifold onto a unified ambient hypersphere and utilizing a localized sequence of ultraspherical (Gegenbauer) polynomials, SKO performs direct integral reconstruction of the target function. Because this localized spherical polynomial kernel is not strictly positive, it bypasses the saturation phenomenon, yielding approximation error bounds that depend strictly on the intrinsic manifold dimension q, rather than the ambient dimension. Furthermore, by formalizing its unnormalized output as an authentic measure support estimator, SKO mathematically decouples the true environmental transition dynamics from the biased observation frequency of the agent. Empirical evaluations confirm that SKO significantly accelerates convergence and outperforms standard attention baselines in autoregressive language modeling.

Related papers

• DyG$^2$T: Modeling Object Dynamics with 3D Gaussian Temporal-Spatial Particle Graph Transformer [86.4618122245946]
  DyG$2$T is a dynamics modeling framework that infers object motion trajectories by spatially completing and temporally discriminating Key Point representations.<n>Experiments on both synthetic and real-world datasets demonstrate that DyG$2$T achieves accurate dynamics modeling and reasoning.
  arXiv  Detail & Related papers  (2026-08-19T03:44:28Z)
• Prototype-Anchored Generalized Manifold Regression for Unknown-Domain Object Detection [58.25418970608328]
  We study Single-Domain Generalized Object Detection (Single-DGOD), which aims to transfer a detector trained on a single source domain to multiple unseen domains.<n>Existing methods mainly rely on simulation-driven strategies, such as data augmentation or textual prompts, to enlarge the training distribution.<n>We propose Manifold Regression with Visual-Text Dual Chain-of-Thought (MR-DCoT), which formulates unknown-domain generalization as a manifold regression problem.
  arXiv  Detail & Related papers  (2026-07-08T09:27:15Z)
• Learning Spatio-Temporal Foundation Models from Pure Synthetic Data [85.23980441738811]
  NeoST is a scalable synthetic-training corpus to real-world bias, a latent-space reasoning architecture that generates and iteratively refines multiple future trajectories, and latent-space objectives that emphasize structural dynamics and enable inference-time correction under distribution shifts.<n>Extensive experiments across diverse real-world benchmarks show that NeoST consistently outperforms existing STFMs in diverse real-worldtemporal systems.
  arXiv  Detail & Related papers  (2026-06-27T02:56:46Z)
• Robust Neural Tucker Factorization with Bias Correction and Adaptive Initialization [1.104960878651584]
  High-dimensional incomplete (HDI) tensors are widely used in traffic and climate applications, but sparse observations make accurate completion difficult.<n>This paper proposes a simple yet effective neural Tucker factorization model with Kaiming initialization and bias correction (KaBiN) for HDI tensor completion.<n> Experiments on three real-world HDI tensor datasets show that KaBiN achieves better performance than the original NeuTucF, while introducing minimal computational overhead.
  arXiv  Detail & Related papers  (2026-06-15T08:17:52Z)
• Nonlocal Bayesian Modeling of Continuous Spatio-Temporal Dynamics [9.926231893220061]
  We propose a principled NonLocal S-NL model (BST) for continuous-temporal fields that learns nonlocal coupling while retaining tractable inference.<n> NLBST represents the latent field via a coordinate-based spatial basis expansion and models the sequential process with a continuous-time ODE.<n>A linear-Gssianau observation model enables Kalmanssian-style sequential sparse updates under missing and irregular observations, while the spatial basis representation enables inductive prediction at unmeasured locations without retraining.
  arXiv  Detail & Related papers  (2026-06-12T09:51:07Z)
• Cohomological Obstructions to Global Counterfactuals: A Sheaf-Theoretic Foundation for Generative Causal Models [7.073238726507493]
  We formalize structural causal models as cellular sheaves over Wasserstein spaces.<n>To ensure tractability, we introduce entropic regularization and derive the Entropic Wasserstein Causal Sheaf Laplacian.<n>Our framework successfully leverages thermodynamic noise to navigate topological barriers in high-dimensional scRNA-seq counterfactuals.
  arXiv  Detail & Related papers  (2026-03-18T06:07:07Z)
• OMNIFLOW: A Physics-Grounded Multimodal Agent for Generalized Scientific Reasoning [51.33849811496781]
  Large Language Models (LLMs) have demonstrated exceptional logical reasoning capabilities but frequently struggle with the continuous dynamics by Partial Differential Equations (PDEs)<n>We propose OMNIFLOW, a multimodal-symbolic architecture designed to ground frozen LLMs in fundamental physical laws without requiring domain-specific parameter updates.<n>We evaluate it on a benchmark spanning microscopic turbulence, theoretical Navier-Stokes, and macroscopic global weather forecasting.
  arXiv  Detail & Related papers  (2026-03-16T18:29:01Z)
• KoopGen: Koopman Generator Networks for Representing and Predicting Dynamical Systems with Continuous Spectra [65.11254608352982]
  We introduce a generator-based neural Koopman framework that models dynamics through a structured, state-dependent representation of Koopman generators.<n>By exploiting the intrinsic Cartesian decomposition into skew-adjoint and self-adjoint components, KoopGen separates conservative transport from irreversible dissipation.
  arXiv  Detail & Related papers  (2026-02-15T06:32:23Z)
• Parallel Complex Diffusion for Scalable Time Series Generation [50.01609741902786]
  PaCoDi is a spectral-native architecture that decouples generative modeling in the frequency domain.<n>We show that PaCoDi outperforms existing baselines in both generation quality and inference speed.
  arXiv  Detail & Related papers  (2026-02-10T14:31:53Z)
• Spatially-informed transformers: Injecting geostatistical covariance biases into self-attention for spatio-temporal forecasting [0.0]
  We propose a hybrid architecture that injects a geostatistic inductive bias directly into the decomposing self-attention mechanism via a learnable costatistics kernel.<n>We demonstrate the phenomenon of Deep Variography'', where the network successfully recovers the true spatial parameters of the underlying process end-to-end via backpropagation.
  arXiv  Detail & Related papers  (2025-12-19T15:32:24Z)
• Foundations of Diffusion Models in General State Spaces: A Self-Contained Introduction [54.95522167029998]
  This article is a self-contained primer on diffusion over general state spaces.<n>We develop the discrete-time view (forward noising via Markov kernels and learned reverse dynamics) alongside its continuous-time limits.<n>A common variational treatment yields the ELBO that underpins standard training losses.
  arXiv  Detail & Related papers  (2025-12-04T18:55:36Z)
• Differential-Integral Neural Operator for Long-Term Turbulence Forecasting [43.04613533979613]
  textbfunderlineDifferential-textbfunderlineIntegral textbfunderlineNeural textbfunderlineOperator (method)<n>method explicitly models the turbulent evolution through parallel branches that learn distinct physical operators.<n>It successfully suppresses error accumulation over hundreds of timesteps, maintains high fidelity in both the vorticity fields and energy spectra, and establishes a new benchmark for physically consistent, long-range turbulence forecast.
  arXiv  Detail & Related papers  (2025-09-25T14:08:26Z)
• A Time-Series Foundation Model by Universal Delay Embedding [4.221753069966852]
  This study introduces Universal Delay Embedding (UDE), a pretrained foundation model designed to revolutionize time-series forecasting.<n>UDE as a dynamical representation of observed data constructs two-dimensional subspace patches from Hankel matrices.<n>In particular, the learned dynamical representations and Koopman operator prediction forms from the patches exhibit exceptional interpretability.
  arXiv  Detail & Related papers  (2025-09-15T16:11:49Z)
• A PAC-Bayesian Perspective on the Interpolating Information Criterion [54.548058449535155]
  We show how a PAC-Bayes bound is obtained for a general class of models, characterizing factors which influence performance in the interpolating regime. We quantify how the test error for overparameterized models achieving effectively zero training error depends on the quality of the implicit regularization imposed by e.g. the combination of model, parameter-initialization scheme.
  arXiv  Detail & Related papers  (2023-11-13T01:48:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.