Semantic Delta: An Interpretable Signal Differentiating Human and LLMs Dialogue

• URL: http://arxiv.org/abs/2603.19849v1
• Date: Fri, 20 Mar 2026 11:01:47 GMT
• Title: Semantic Delta: An Interpretable Signal Differentiating Human and LLMs Dialogue
• Abstract summary: We introduce a lightweight metric derived from semantic categories distribution.<n>We define semantic delta as the difference between the two most dominant category intensities within a dialogue.<n>Results show that AI-generated texts consistently produce higher deltas than human texts.
• Score: 3.9431873099460826
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Do LLMs talk like us? This question intrigues a multitude of scholar and it is relevant in many fields, from education to academia. This work presents an interpretable statistical feature for distinguishing human written and LLMs generated dialogue. We introduce a lightweight metric derived from semantic categories distribution. Using the Empath lexical analysis framework, each text is mapped to a set of thematic intensity scores. We define semantic delta as the difference between the two most dominant category intensities within a dialogue, hypothesizing that LLM outputs exhibit stronger thematic concentration than human discourse. To evaluate this hypothesis, conversational data were generated from multiple LLM configurations and compared against heterogeneous human corpora, including scripted dialogue, literary works, and online discussions. A Welch t-test was applied to the resulting distributions of semantic delta values. Results show that AI-generated texts consistently produce higher deltas than human texts, indicating a more rigid topics structure, whereas human dialogue displays a broader and more balanced semantic spread. Rather than replacing existing detection techniques, the proposed zero-shot metric provides a computationally inexpensive complementary signal that can be integrated into ensemble detection systems. These finding also contribute to the broader empirical understanding of LLM behavioural mimicry and suggest that thematic distribution constitutes a quantifiable dimension along which current models fall short of human conversational dynamics.

Related papers

• Saying More Than They Know: A Framework for Quantifying Epistemic-Rhetorical Miscalibration in Large Language Models [0.30458514384586394]
  Large language models (LLMs) exhibit systematic miscalibration with rhetorical intensity not proportionate to epistemic grounding.<n>This study proposes a framework for quantifying this decoupling by designing a triadic epistemic-rhetorical marker (ERM) taxonomy.<n> applied to 225 argumentative texts spanning approximately 0.6 Million tokens across human expert, human non-expert, and LLM-generated sub-corpora.
  arXiv  Detail & Related papers  (2026-03-27T05:33:10Z)
• Decoding AI Authorship: Can LLMs Truly Mimic Human Style Across Literature and Politics? [0.0]
  This study investigates the ability of state-of-the-art large language models (LLMs) to emulate the authorial signatures of prominent literary and political figures.<n>We generated synthetic corpora evaluated through a complementary framework combining transformer-based classification (BERT) and interpretable machine learning (XGBoost)<n>Our methodology integrates Linguistic Inquiry and Word Count markers, perplexity, and readability indices to assess the divergence between AI-generated and human-authored text.
  arXiv  Detail & Related papers  (2026-03-24T13:58:09Z)
• Simulating Meaning, Nevermore! Introducing ICR: A Semiotic-Hermeneutic Metric for Evaluating Meaning in LLM Text Summaries [13.104408059072457]
  This article proposes an interdisciplinary framework for studying meaning in large language model (LLM) generated language.<n>We review prior scholarship on meaning and machines, examining how linguistic signs are transformed into vectorized representations in static and contextualized embedding models.<n>We then introduce the Inductive Conceptual Rating (ICR) metric, designed to assess semantic accuracy and meaning alignment in LLM-outputs.
  arXiv  Detail & Related papers  (2026-02-03T18:05:09Z)
• EMODIS: A Benchmark for Context-Dependent Emoji Disambiguation in Large Language Models [6.145223237741804]
  Large language models (LLMs) are increasingly deployed in real-world communication settings, yet their ability to resolve context-dependent ambiguity remains underexplored.<n>We present EMODIS, a new benchmark for evaluating LLMs' capacity to interpret ambiguous emoji expressions under minimal but contrastive textual contexts.
  arXiv  Detail & Related papers  (2025-11-10T15:24:01Z)
• Computational Turing Test Reveals Systematic Differences Between Human and AI Language [0.0]
  Large language models (LLMs) are increasingly used in the social sciences to simulate human behavior.<n>Existing validation efforts rely heavily on human-judgment-based evaluations.<n>This paper introduces a computational Turing test to assess how closely LLMs approximate human language.
  arXiv  Detail & Related papers  (2025-11-06T08:56:37Z)
• QUDsim: Quantifying Discourse Similarities in LLM-Generated Text [70.22275200293964]
  We introduce an abstraction based on linguistic theories in Questions Under Discussion (QUD) and question semantics to help quantify differences in discourse progression.<n>We then use this framework to build $textbfQUDsim$, a similarity metric that can detect discursive parallels between documents.<n>Using QUDsim, we find that LLMs often reuse discourse structures (more so than humans) across samples, even when content differs.
  arXiv  Detail & Related papers  (2025-04-12T23:46:09Z)
• The dynamics of meaning through time: Assessment of Large Language Models [2.5864824580604515]
  This study aims to evaluate the capabilities of various large language models (LLMs) in capturing temporal dynamics of meaning.<n>Our comparative analysis includes prominent models like ChatGPT, GPT-4, Claude, Bard, Gemini, and Llama.<n>Findings reveal marked differences in each model's handling of historical context and semantic shifts, highlighting both strengths and limitations in temporal semantic understanding.
  arXiv  Detail & Related papers  (2025-01-09T19:56:44Z)
• Human Variability vs. Machine Consistency: A Linguistic Analysis of Texts Generated by Humans and Large Language Models [0.0]
  We identify significant differences between human-written texts and those generated by large language models (LLMs)<n>Our findings indicate that humans write texts that are less cognitively demanding, with higher semantic content, and richer emotional content compared to texts generated by LLMs.
  arXiv  Detail & Related papers  (2024-12-04T04:38:35Z)
• Categorical Syllogisms Revisited: A Review of the Logical Reasoning Abilities of LLMs for Analyzing Categorical Syllogism [62.571419297164645]
  This paper provides a systematic overview of prior works on the logical reasoning ability of large language models for analyzing categorical syllogisms.<n>We first investigate all the possible variations for the categorical syllogisms from a purely logical perspective.<n>We then examine the underlying configurations (i.e., mood and figure) tested by the existing datasets.
  arXiv  Detail & Related papers  (2024-06-26T21:17:20Z)
• Scaling Data Diversity for Fine-Tuning Language Models in Human Alignment [84.32768080422349]
  Alignment with human preference prevents large language models from generating misleading or toxic content. We propose a new formulation of prompt diversity, implying a linear correlation with the final performance of LLMs after fine-tuning.
  arXiv  Detail & Related papers  (2024-03-17T07:08:55Z)
• Syntax and Semantics Meet in the "Middle": Probing the Syntax-Semantics Interface of LMs Through Agentivity [68.8204255655161]
  We present the semantic notion of agentivity as a case study for probing such interactions. This suggests LMs may potentially serve as more useful tools for linguistic annotation, theory testing, and discovery.
  arXiv  Detail & Related papers  (2023-05-29T16:24:01Z)
• A Vector Quantized Approach for Text to Speech Synthesis on Real-World Spontaneous Speech [94.64927912924087]
  We train TTS systems using real-world speech from YouTube and podcasts. Recent Text-to-Speech architecture is designed for multiple code generation and monotonic alignment. We show thatRecent Text-to-Speech architecture outperforms existing TTS systems in several objective and subjective measures.
  arXiv  Detail & Related papers  (2023-02-08T17:34:32Z)
• Contextualized Semantic Distance between Highly Overlapped Texts [85.1541170468617]
  Overlapping frequently occurs in paired texts in natural language processing tasks like text editing and semantic similarity evaluation. This paper aims to address the issue with a mask-and-predict strategy. We take the words in the longest common sequence as neighboring words and use masked language modeling (MLM) to predict the distributions on their positions. Experiments on Semantic Textual Similarity show NDD to be more sensitive to various semantic differences, especially on highly overlapped paired texts.
  arXiv  Detail & Related papers  (2021-10-04T03:59:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.