The Last Fingerprint: How Markdown Training Shapes LLM Prose

• URL: http://arxiv.org/abs/2603.27006v1
• Date: Fri, 27 Mar 2026 21:42:06 GMT
• Title: The Last Fingerprint: How Markdown Training Shapes LLM Prose
• Abstract summary: We propose that the em dash is markdown leaking into prose.<n>We present a five-step genealogy connecting training data composition, structural internalization, the dual-register status of the em dash, and post-training amplification.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models produce em dashes at varying rates, and the observation that some models "overuse" them has become one of the most widely discussed markers of AI-generated text. Yet no mechanistic account of this pattern exists, and the parallel observation that LLMs default to markdown-formatted output has never been connected to it. We propose that the em dash is markdown leaking into prose -- the smallest surviving unit of the structural orientation that LLMs acquire from markdown-saturated training corpora. We present a five-step genealogy connecting training data composition, structural internalization, the dual-register status of the em dash, and post-training amplification. We test this with a two-condition suppression experiment across twelve models from five providers (Anthropic, OpenAI, Meta, Google, DeepSeek): when models are instructed to avoid markdown formatting, overt features (headers, bullets, bold) are eliminated or nearly eliminated, but em dashes persist -- except in Meta's Llama models, which produce none at all. Em dash frequency and suppression resistance vary from 0.0 per 1,000 words (Llama) to 9.1 (GPT-4.1 under suppression), functioning as a signature of the specific fine-tuning procedure applied. A three-condition suppression gradient shows that even explicit em dash prohibition fails to eliminate the artifact in some models, and a base-vs-instruct comparison confirms that the latent tendency exists pre-RLHF. These findings connect two previously isolated online discourses and reframe em dash frequency as a diagnostic of fine-tuning methodology rather than a stylistic defect.

Related papers

• DriftScope: Measuring The Hidden Effects of Diffusion Model Adaptation [50.584834483403675]
  Adapting pre-trained text-to-image diffusion models, whether to learn new visual concepts or erase unwanted ones, is routinely evaluated on its intended effects alone.<n>We demonstrate that adaptation systematically damages semantically unrelated concepts in ways that aggregate metrics structurally cannot surface.<n>We introduce DriftScope, a prompt-level diagnostic tool that takes any two model checkpoints and returns a ranked list of tokens whose visual concepts have shifted most between them.
  arXiv  Detail & Related papers  (2026-06-30T20:57:51Z)
• Verified Detection and Prevention of Concurrency Anomalies in Multi-Agent Large Language Model Systems [0.0]
  Multi-agent LLM systems state through memory stores, vector indices, and tool registries.<n>We model such sharing as long-running read-generate-write operations under deterministic-generation semantics.<n>We formalize four anomalies in TLA+: stale-generation, phantom-tool, causal-cascade, and tool-effect reordering.
  arXiv  Detail & Related papers  (2026-06-15T18:19:34Z)
• Hacking Generative Perplexity: Why Unconditional Text Evaluation Needs Distributional Metrics [49.443264461057645]
  Diffusion and continuous flow-based language models have emerged as the leading non-autoregressive alternatives to language modeling.<n>Progress in both paradigms is overwhelmingly tracked by generative perplexity (gen-PPL): the per-token negative log-likelihood of samples under a frozen autoregressive (AR) scorer.<n>We argue that this metric is unsound. By construction, gen-PPL measures only predictability under the scoring AR, not grammaticality or semantic coherence.
  arXiv  Detail & Related papers  (2026-06-07T02:35:56Z)
• RULER: Representation-Level Verification of Machine Unlearning [0.17188280334580197]
  We introduce RULER, a set of representation-level verification metrics.<n>M2 measures whether forget-set records occupy the same representational position as in a model retrained without them.<n>M4 detects residuals from the unlearned model's internal similarity structure alone, without retraining.
  arXiv  Detail & Related papers  (2026-05-26T18:41:48Z)
• Judge Circuits [24.717818484378583]
  We investigate the internal mechanism in Gemma-3, Qwen2.5, and Llama-3.<n>We find that judgments across structured understanding and open-ended preference tasks share a sparse, generalized Latent Evaluator sub-graph.<n>We provide a mechanistic account of format-induced inconsistency on the open-weight models we study.
  arXiv  Detail & Related papers  (2026-05-15T14:57:21Z)
• Forgetting That Sticks: Quantization-Permanent Unlearning via Circuit Attribution [3.6704226968275253]
  We show gradient-based methods that achieve meaningful forgetting lose it under compression, while methods that survive quantization barely change the model.<n>We present MANSU, which resolves both modes by combining causal circuit attribution to isolate the minimal forget-set subgraph.<n>We additionally introduce Circuit Attribution Divergence (CAD), a mechanistic verification metric distinguishing structural erasure from behavioral suppression.
  arXiv  Detail & Related papers  (2026-05-14T17:44:10Z)
• From Early Encoding to Late Suppression: Interpreting LLMs on Character Counting Tasks [49.57538588967748]
  Large language models (LLMs) exhibit failures on elementary symbolic tasks such as character counting in a word, despite excelling on complex benchmarks.<n>We uncover a consistent phenomenon across modern architectures, including LLaMA, Qwen, and Gemma.<n>We show that symbolic reasoning failures in LLMs are not due to missing representations or insufficient scale, but arise from structured interference within the model's computation graph.
  arXiv  Detail & Related papers  (2026-04-01T11:40:12Z)
• BTZSC: A Benchmark for Zero-Shot Text Classification Across Cross-Encoders, Embedding Models, Rerankers and LLMs [0.0]
  Zero-shot text classification (ZSC) offers the promise of eliminating costly task-specific annotation.<n>Recent advances in text-embedding models, rerankers, and instruction-tuned large language models (LLMs) have challenged the dominance of NLI-based architectures.<n>We introduce BTZSC, a comprehensive benchmark of 22 public datasets spanning sentiment, topic, intent, and emotion classification.
  arXiv  Detail & Related papers  (2026-03-12T14:43:20Z)
• Semantics as a Shield: Label Disguise Defense (LDD) against Prompt Injection in LLM Sentiment Classification [5.963719408944521]
  This paper introduces Label Disguise Defense (LDD), a lightweight strategy that conceals true labels by replacing them with semantically transformed alias labels.<n>We evaluate LDD across nine state-of-the-art models, including GPT-5, GPT-4o, LLaMA3.2, Gemma3, and Mistral variants.
  arXiv  Detail & Related papers  (2025-11-23T20:16:51Z)
• Antislop: A Comprehensive Framework for Identifying and Eliminating Repetitive Patterns in Language Models [8.02516998509823]
  We present Antislop, a framework providing tools to both detect and eliminate these overused patterns.<n>The Antislop Sampler uses backtracking to suppress unwanted strings at inference time without destroying vocabulary.<n> FTPO achieves 90% slop reduction while maintaining or improving performance in cross-domain evals including GSM8K, MMLU, and creative writing tasks.
  arXiv  Detail & Related papers  (2025-10-16T18:22:22Z)
• Dual-Stage Reweighted MoE for Long-Tailed Egocentric Mistake Detection [85.0189917888094]
  We propose a Dual-Stage Reweighted Mixture-of-Experts (DR-MoE) framework to handle the challenges posed by subtle and infrequent mistakes.<n>The proposed method achieves strong performance, particularly in identifying rare and ambiguous mistake instances.
  arXiv  Detail & Related papers  (2025-09-16T12:00:42Z)
• Predicting the Performance of Black-box LLMs through Self-Queries [60.87193950962585]
  Large language models (LLMs) are increasingly relied on in AI systems, predicting when they make mistakes is crucial.<n>In this paper, we extract features of LLMs in a black-box manner by using follow-up prompts and taking the probabilities of different responses as representations.<n>We demonstrate that training a linear model on these low-dimensional representations produces reliable predictors of model performance at the instance level.
  arXiv  Detail & Related papers  (2025-01-02T22:26:54Z)
• Extract Free Dense Misalignment from CLIP [7.0247398611254175]
  This work proposes a novel approach, dubbed CLIP4DM, for detecting dense misalignments from pre-trained CLIP.<n>We revamp the gradient-based attribution computation method, enabling negative gradient of individual text tokens to indicate misalignment.<n>Our method demonstrates state-of-the-art performance among zero-shot models and competitive performance with fine-tuned models.
  arXiv  Detail & Related papers  (2024-12-24T12:51:05Z)
• Non-Halting Queries: Exploiting Fixed Points in LLMs [4.091772241106195]
  We introduce a new vulnerability that exploits fixed points in autoregressive models and use it to craft queries that never halt.<n>We rigorously analyze the conditions under which the non-halting anomaly presents itself.<n>We demonstrate non-halting queries in many experiments performed in base unaligned models.
  arXiv  Detail & Related papers  (2024-10-08T18:38:32Z)
• Autoregressive Speech Synthesis without Vector Quantization [135.4776759536272]
  We present MELLE, a novel continuous-valued token based language modeling approach for text-to-speech synthesis (TTS)<n>MELLE autoregressively generates continuous mel-spectrogram frames directly from text condition.<n>MELLE mitigates robustness issues by avoiding the inherent flaws of sampling vector-quantized codes.
  arXiv  Detail & Related papers  (2024-07-11T14:36:53Z)
• Disperse-Then-Merge: Pushing the Limits of Instruction Tuning via Alignment Tax Reduction [75.25114727856861]
  Large language models (LLMs) tend to suffer from deterioration at the latter stage ofSupervised fine-tuning process. We introduce a simple disperse-then-merge framework to address the issue. Our framework outperforms various sophisticated methods such as data curation and training regularization on a series of standard knowledge and reasoning benchmarks.
  arXiv  Detail & Related papers  (2024-05-22T08:18:19Z)
• A Transformer Model for Boundary Detection in Continuous Sign Language [55.05986614979846]
  The Transformer model is employed for both Isolated Sign Language Recognition and Continuous Sign Language Recognition. The training process involves using isolated sign videos, where hand keypoint features extracted from the input video are enriched. The trained model, coupled with a post-processing method, is then applied to detect isolated sign boundaries within continuous sign videos.
  arXiv  Detail & Related papers  (2024-02-22T17:25:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.