Saying More Than They Know: A Framework for Quantifying Epistemic-Rhetorical Miscalibration in Large Language Models

• URL: http://arxiv.org/abs/2604.19768v1
• Date: Fri, 27 Mar 2026 05:33:10 GMT
• Title: Saying More Than They Know: A Framework for Quantifying Epistemic-Rhetorical Miscalibration in Large Language Models
• Abstract summary: Large language models (LLMs) exhibit systematic miscalibration with rhetorical intensity not proportionate to epistemic grounding.<n>This study proposes a framework for quantifying this decoupling by designing a triadic epistemic-rhetorical marker (ERM) taxonomy.<n> applied to 225 argumentative texts spanning approximately 0.6 Million tokens across human expert, human non-expert, and LLM-generated sub-corpora.
• Score: 0.30458514384586394
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) exhibit systematic miscalibration with rhetorical intensity not proportionate to epistemic grounding. This study tests this hypothesis and proposes a framework for quantifying this decoupling by designing a triadic epistemic-rhetorical marker (ERM) taxonomy. The taxonomy is operationalized through composite metrics of form-meaning divergence (FMD), genuine-to-performed epistemic ratio (GPR), and rhetorical device distribution entropy (RDDE). Applied to 225 argumentative texts spanning approximately 0.6 Million tokens across human expert, human non-expert, and LLM-generated sub-corpora, the framework identifies a consistent, model-agnostic LLM epistemic signature. LLM-generated texts produce tricolon at nearly twice the expert rate ($Δ= 0.95$), while human authors produce erotema at more than twice the LLM rate. Performed hesitancy markers appear at twice the human density in LLM output. FMD is significantly elevated in LLM texts relative to both human groups ($p < 0.001, Δ= 0.68$), and rhetorical devices are distributed significantly more uniformly across LLM documents. The findings are consistent with theoretical intuitions derived from Gricean pragmatics, Relevance Theory, and Brandomian inferentialism. The annotation pipeline is fully automatable, making it deployable as a lightweight screening tool for epistemic miscalibration in AI-generated content and as a theoretically motivated feature set for LLM-generated text detection pipelines.

Related papers

• Artificial Intelligence Can Match Domain Experts in Evidence Extraction and Critical Appraisal of Microbial Oncogenesis Research Publications [4.342264393386239]
  Methodological appraisal and identification of contradictions within full-texts were the most persistent LLM vulnerabilities.<n>LLM responses aligned closely with experts, with GPT-5 and GPT-5 Nano achieving score distributions indistinguishable from experts.
  arXiv  Detail & Related papers  (2026-08-07T14:07:57Z)
• Influence-Guided Symbolic Regression: Scientific Discovery via LLM-Driven Equation Search with Granular Feedback [56.69850045068714]
  We introduce textitInfluence-Guided Regression (IGSR), a method that frames equation discovery as an iterative two-step process.<n>We demonstrate IGSRs effectiveness on a diverse suite of benchmarks, including LLM-SRBench, pharmacological PKPD models, an epidemiological simulation, and real-world genomic data.
  arXiv  Detail & Related papers  (2026-05-27T23:48:01Z)
• Multi-Task LLM with LoRA Fine-Tuning for Automated Cancer Staging and Biomarker Extraction [6.53191669778827]
  This study introduces a parameter-efficient, multi-task framework for automating the extraction of Tumor-Node-Metastasis (TNM) staging, histologic grade, and biomarkers.<n>We fine-tune a Llama-3-8B-Instruct encoder using Low-Rank Adaptation (LoRA) on a curated, expert-verified dataset of 10,677 reports.
  arXiv  Detail & Related papers  (2026-04-14T22:26:30Z)
• Semantic Delta: An Interpretable Signal Differentiating Human and LLMs Dialogue [3.9431873099460826]
  We introduce a lightweight metric derived from semantic categories distribution.<n>We define semantic delta as the difference between the two most dominant category intensities within a dialogue.<n>Results show that AI-generated texts consistently produce higher deltas than human texts.
  arXiv  Detail & Related papers  (2026-03-20T11:01:47Z)
• Mitigating Hallucinations in Zero-Shot Scientific Summarisation: A Pilot Study [0.0]
  Large language models (LLMs) produce context inconsistency hallucinations.<n>This research project investigates whether prompt engineering methods can mitigate context inconsistency in zero-shot scientific summarisation tasks.
  arXiv  Detail & Related papers  (2025-11-30T15:19:41Z)
• Framework for Machine Evaluation of Reasoning Completeness in Large Language Models For Classification Tasks [0.0]
  This paper introduces RACE-Reasoning Alignment for Completeness of Explanations.<n>We analyze four widely used text classification datasets-WIKI ONTOLOGY, AG NEWS, IMDB, and GOEMOTIONS.<n>We show that correct predictions exhibit higher coverage of supporting features, while incorrect predictions are associated with elevated coverage of contradicting features.
  arXiv  Detail & Related papers  (2025-10-23T20:22:22Z)
• How Do LLM-Generated Texts Impact Term-Based Retrieval Models? [76.92519309816008]
  This paper investigates the influence of large language models (LLMs) on term-based retrieval models.<n>Our linguistic analysis reveals that LLM-generated texts exhibit smoother high-frequency and steeper low-frequency Zipf slopes.<n>Our study further explores whether term-based retrieval models demonstrate source bias, concluding that these models prioritize documents whose term distributions closely correspond to those of the queries.
  arXiv  Detail & Related papers  (2025-08-25T06:43:27Z)
• "I know myself better, but not really greatly": How Well Can LLMs Detect and Explain LLM-Generated Texts? [10.454446545249096]
  This paper investigates detection and explanation capabilities of current LLMs across two settings: binary (human vs. LLM-generated) and ternary classification (including an undecided'' class)<n>We evaluate 6 close- and open-source LLMs of varying sizes and find that self-detection (LLMs identifying their own outputs) consistently outperforms cross-detection (identifying outputs from other LLMs)<n>Our findings underscore the limitations of current LLMs in self-detection and self-explanation, highlighting the need for further research to address overfitting and enhance generalizability.
  arXiv  Detail & Related papers  (2025-02-18T11:00:28Z)
• LLM2: Let Large Language Models Harness System 2 Reasoning [65.89293674479907]
  Large language models (LLMs) have exhibited impressive capabilities across a myriad of tasks, yet they occasionally yield undesirable outputs.<n>We introduce LLM2, a novel framework that combines an LLM with a process-based verifier.<n>LLMs2 is responsible for generating plausible candidates, while the verifier provides timely process-based feedback to distinguish desirable and undesirable outputs.
  arXiv  Detail & Related papers  (2024-12-29T06:32:36Z)
• Evaluating Human Alignment and Model Faithfulness of LLM Rationale [66.75309523854476]
  We study how well large language models (LLMs) explain their generations through rationales. We show that prompting-based methods are less "faithful" than attribution-based explanations.
  arXiv  Detail & Related papers  (2024-06-28T20:06:30Z)
• A Theory for Token-Level Harmonization in Retrieval-Augmented Generation [76.75124161306795]
  Retrieval-augmented generation (RAG) utilizes retrieved texts to enhance large language models (LLMs)<n>This paper provides a theory to explain and trade off the benefit and detriment in RAG.<n>Based on our theory, we propose a practical novel method, Tok-RAG, which achieves collaborative generation between the pure LLM and RAG.
  arXiv  Detail & Related papers  (2024-06-03T02:56:14Z)
• Zero-shot Causal Graph Extrapolation from Text via LLMs [50.596179963913045]
  We evaluate the ability of large language models (LLMs) to infer causal relations from natural language. LLMs show competitive performance in a benchmark of pairwise relations without needing (explicit) training samples. We extend our approach to extrapolating causal graphs through iterated pairwise queries.
  arXiv  Detail & Related papers  (2023-12-22T13:14:38Z)
• Sentiment Analysis through LLM Negotiations [58.67939611291001]
  A standard paradigm for sentiment analysis is to rely on a singular LLM and makes the decision in a single round. This paper introduces a multi-LLM negotiation framework for sentiment analysis.
  arXiv  Detail & Related papers  (2023-11-03T12:35:29Z)
• On the Relation between Internal Language Model and Sequence Discriminative Training for Neural Transducers [52.88268942796418]
  Internal language model (ILM) subtraction has been widely applied to improve the performance of the RNN-Transducer. We show that sequence discriminative training has a strong correlation with ILM subtraction from both theoretical and empirical points of view.
  arXiv  Detail & Related papers  (2023-09-25T13:35:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.