Scale-Dependent Semantic Dynamics Revealed by Allan Deviation

• URL: http://arxiv.org/abs/2601.21678v1
• Date: Thu, 29 Jan 2026 13:10:59 GMT
• Title: Scale-Dependent Semantic Dynamics Revealed by Allan Deviation
• Authors: Debayan Dasgupta,
• Abstract summary: We analyze the stability of meaning by treating ordered sentence embeddings as a displacement signal.<n>We find that while large language models successfully mimic the local scaling statistics of human text, they exhibit a systematic reduction in their stability horizon.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While language progresses through a sequence of semantic states, the underlying dynamics of this progression remain elusive. Here, we treat the semantic progression of written text as a stochastic trajectory in a high-dimensional state space. We utilize Allan deviation, a tool from precision metrology, to analyze the stability of meaning by treating ordered sentence embeddings as a displacement signal. Our analysis reveals two distinct dynamical regimes: short-time power-law scaling, which differentiates creative literature from technical texts, and a long-time crossover to a stability-limited noise floor. We find that while large language models successfully mimic the local scaling statistics of human text, they exhibit a systematic reduction in their stability horizon. These results establish semantic coherence as a measurable physical property, offering a framework to differentiate the nuanced dynamics of human cognition from the patterns generated by algorithmic models.

Related papers

• Dynamical Systems Analysis Reveals Functional Regimes in Large Language Models [0.8694591156258423]
  Large language models perform text generation through high-dimensional internal dynamics.<n>Most interpretability approaches emphasise static representations or causal interventions, leaving temporal structure largely unexplored.<n>We discuss a composite dynamical metric, computed from activation time-series during autoregressive generation.
  arXiv  Detail & Related papers  (2026-01-11T21:57:52Z)
• Stable Language Guidance for Vision-Language-Action Models [62.80963701282789]
  Residual Semantic Steering is a probabilistic framework that disentangles physical affordance from semantic execution.<n> RSS achieves state-of-the-art robustness, maintaining performance even under adversarial linguistic perturbations.
  arXiv  Detail & Related papers  (2026-01-07T16:16:10Z)
• LyTimeT: Towards Robust and Interpretable State-Variable Discovery [7.505092370141079]
  LyTimeT is a framework for interpretable variable extraction.<n>It learns robust and stable latent representations of dynamical video systems.<n>Our results demonstrate that combiningtemporal attention with stability constraints yields predictive models.
  arXiv  Detail & Related papers  (2025-10-22T16:03:10Z)
• Discovering dynamical laws for speech gestures [0.0]
  We discover models in the form of symbolic equations that govern articulatory gestures during speech.<n>A sparse symbolic regression algorithm is used to discover models from kinematic data on the tongue and lips.
  arXiv  Detail & Related papers  (2025-04-07T09:03:32Z)
• Unraveling Text Generation in LLMs: A Stochastic Differential Equation Approach [3.4039202831583903]
  This paper explores the application of Differential Equations (SDE) to interpret the text generation process of Large Language Models (LLMs) such as GPT-4. We represent this generation process using SDE to capture both deterministic trends and perturbations. We fit these functions using neural networks and validate the model on real-world text corpora.
  arXiv  Detail & Related papers  (2024-08-17T15:30:27Z)
• Physically Analyzable AI-Based Nonlinear Platoon Dynamics Modeling During Traffic Oscillation: A Koopman Approach [4.379212829795889]
  There exists a critical need for a modeling methodology with high accuracy while concurrently achieving physical analyzability. This paper proposes an AI-based Koopman approach to model the unknown nonlinear platoon dynamics harnessing the power of AI.
  arXiv  Detail & Related papers  (2024-06-20T19:35:21Z)
• Vocabulary-Defined Semantics: Latent Space Clustering for Improving In-Context Learning [32.178931149612644]
  In-context learning enables language models to adapt to downstream data or incorporate tasks by few samples as demonstrations within the prompts. However, the performance of in-context learning can be unstable depending on the quality, format, or order of demonstrations. We propose a novel approach "vocabulary-defined semantics"
  arXiv  Detail & Related papers  (2024-01-29T14:29:48Z)
• PoetryDiffusion: Towards Joint Semantic and Metrical Manipulation in Poetry Generation [58.36105306993046]
  Controllable text generation is a challenging and meaningful field in natural language generation (NLG) In this paper, we pioneer the use of the Diffusion model for generating sonnets and Chinese SongCi poetry. Our model outperforms existing models in automatic evaluation of semantic, metrical, and overall performance as well as human evaluation.
  arXiv  Detail & Related papers  (2023-06-14T11:57:31Z)
• Incorporating Dynamic Semantics into Pre-Trained Language Model for Aspect-based Sentiment Analysis [67.41078214475341]
  We propose Dynamic Re-weighting BERT (DR-BERT) to learn dynamic aspect-oriented semantics for ABSA. Specifically, we first take the Stack-BERT layers as a primary encoder to grasp the overall semantic of the sentence. We then fine-tune it by incorporating a lightweight Dynamic Re-weighting Adapter (DRA)
  arXiv  Detail & Related papers  (2022-03-30T14:48:46Z)
• A comprehensive comparative evaluation and analysis of Distributional Semantic Models [61.41800660636555]
  We perform a comprehensive evaluation of type distributional vectors, either produced by static DSMs or obtained by averaging the contextualized vectors generated by BERT. The results show that the alleged superiority of predict based models is more apparent than real, and surely not ubiquitous. We borrow from cognitive neuroscience the methodology of Representational Similarity Analysis (RSA) to inspect the semantic spaces generated by distributional models.
  arXiv  Detail & Related papers  (2021-05-20T15:18:06Z)
• Mechanisms for Handling Nested Dependencies in Neural-Network Language Models and Humans [75.15855405318855]
  We studied whether a modern artificial neural network trained with "deep learning" methods mimics a central aspect of human sentence processing. Although the network was solely trained to predict the next word in a large corpus, analysis showed the emergence of specialized units that successfully handled local and long-distance syntactic agreement. We tested the model's predictions in a behavioral experiment where humans detected violations in number agreement in sentences with systematic variations in the singular/plural status of multiple nouns.
  arXiv  Detail & Related papers  (2020-06-19T12:00:05Z)
• Euclideanizing Flows: Diffeomorphic Reduction for Learning Stable Dynamical Systems [74.80320120264459]
  We present an approach to learn such motions from a limited number of human demonstrations. The complex motions are encoded as rollouts of a stable dynamical system. The efficacy of this approach is demonstrated through validation on an established benchmark as well demonstrations collected on a real-world robotic system.
  arXiv  Detail & Related papers  (2020-05-27T03:51:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.