Do Language Models Encode Knowledge of Linguistic Constraint Violations?

• URL: http://arxiv.org/abs/2605.12055v2
• Date: Thu, 14 May 2026 07:07:27 GMT
• Title: Do Language Models Encode Knowledge of Linguistic Constraint Violations?
• Abstract summary: Large Language Models (LLMs) achieve strong linguistic performance, yet their internal mechanisms for producing these predictions remain unclear.<n>We investigate the hypothesis that LLMs encode representations of linguistic constraint violations within their parameters, which are selectively activated when processing ungrammatical sentences.<n>We introduce a sensitivity score for identifying features that are preferentially activated on constraint-violated versus well-formed inputs, enabling unsupervised detection of potential violation-specific features.
• Score: 6.1335559884409685
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) achieve strong linguistic performance, yet their internal mechanisms for producing these predictions remain unclear. We investigate the hypothesis that LLMs encode representations of linguistic constraint violations within their parameters, which are selectively activated when processing ungrammatical sentences. To test this, we use sparse autoencoders to decompose polysemantic activations into sparse, monosemantic features and recover candidates for violation-related features. We introduce a sensitivity score for identifying features that are preferentially activated on constraint-violated versus well-formed inputs, enabling unsupervised detection of potential violation-specific features. We further propose a conjunctive falsification framework with three criteria evaluated jointly. Overall, the results are negative in two respects: (1) the falsification criteria are not jointly satisfied across linguistic phenomena, and (2) no features are consistently shared across all categories. While some phenomena show partial evidence of selective causal structure, the overall pattern provides limited support for a unified set of grammatical violation detectors in current LMs.

Related papers

• ClarifyCodeBench: Evaluating LLMs on Clarifying Ambiguous Requirements for Code Generation [54.788849448970154]
  We introduce ClarifyCodeBench, a novel interactive benchmark for evaluating Large Language Models' capability in resolving requirement ambiguity.<n>We conduct a systematic evaluation of six state-of-the-art LLMs using ClarifyCodeBench.<n>Our empirical results yield three critical insights: 1) Capability Decoupling: Strong code generation performance does not inherently translate to effective requirement clarification; 2) The Reasoning Paradox: While increased computational thinking enhances code correctness, it yields marginal gains in identifying ambiguities.
  arXiv  Detail & Related papers  (2026-07-01T09:58:59Z)
• Automated Detection and Classification of Delusion-related Content in Naturalistic Audio Diaries Using Multi-Agent Language Models [9.896454972048376]
  Large language models (LLMs) offer new possibilities for automated mental illness phenomenology.<n>We present a novel automated, multi-agent LLM pipeline for the fine-grained, multi-label extraction of language suggestive of delusional beliefs.<n>This work provides a validated and scalable pipeline for the automated detection and characterization of content suggesting delusional beliefs in naturalistic speech.
  arXiv  Detail & Related papers  (2026-05-23T22:18:40Z)
• Multilingual Cognitive Impairment Detection in the Era of Foundation Models [6.141505279958125]
  We evaluate cognitive impairment (CI) classification from transcripts of speech in English, Slovene, and Korean.<n>We compare zero-shot large language models (LLMs) used as direct classifiers under three input settings.
  arXiv  Detail & Related papers  (2026-04-08T07:22:43Z)
• Do Sparse Autoencoders Identify Reasoning Features in Language Models? [12.693974363520423]
  We investigate whether sparse autoencoders (SAEs) identify genuine reasoning features in large language models (LLMs)<n>We first show through a simple theoretical analysis that $ell_$-regularized SAEs are intrinsically biased toward low-dimensional patterns.<n>Motivated by this bias, we introduce a falsification-oriented evaluation framework to test whether feature activation reflects reasoning processes or superficial linguistic correlates.
  arXiv  Detail & Related papers  (2026-01-09T09:54:36Z)
• From Disagreement to Understanding: The Case for Ambiguity Detection in NLI [3.5164356851161984]
  This position paper argues that annotation disagreement in Natural Language Inference (NLI) is not mere noise but often reflects meaningful variation.<n>We call for a shift toward ambiguity-aware NLI that first identifies ambiguous input pairs, classifies their types, and only then proceeds to inference.
  arXiv  Detail & Related papers  (2025-07-20T20:27:35Z)
• Understanding Refusal in Language Models with Sparse Autoencoders [27.212781538459588]
  We use sparse autoencoders to identify latent features that causally mediate refusal behaviors.<n>We intervene on refusal-related features to assess their influence on generation.<n>This enables a fine-grained inspection of how refusal manifests at the activation level.
  arXiv  Detail & Related papers  (2025-05-29T15:33:39Z)
• Revisiting subword tokenization: A case study on affixal negation in large language models [57.75279238091522]
  We measure the impact of affixal negation on modern English large language models (LLMs) We conduct experiments using LLMs with different subword tokenization methods. We show that models can, on the whole, reliably recognize the meaning of affixal negation.
  arXiv  Detail & Related papers  (2024-04-03T03:14:27Z)
• Language models are not naysayers: An analysis of language models on negation benchmarks [58.32362243122714]
  We evaluate the ability of current-generation auto-regressive language models to handle negation. We show that LLMs have several limitations including insensitivity to the presence of negation, an inability to capture the lexical semantics of negation, and a failure to reason under negation.
  arXiv  Detail & Related papers  (2023-06-14T01:16:37Z)
• Simple Linguistic Inferences of Large Language Models (LLMs): Blind Spots and Blinds [59.71218039095155]
  We evaluate language understanding capacities on simple inference tasks that most humans find trivial. We target (i) grammatically-specified entailments, (ii) premises with evidential adverbs of uncertainty, and (iii) monotonicity entailments. The models exhibit moderate to low performance on these evaluation sets.
  arXiv  Detail & Related papers  (2023-05-24T06:41:09Z)
• We're Afraid Language Models Aren't Modeling Ambiguity [136.8068419824318]
  Managing ambiguity is a key part of human language understanding. We characterize ambiguity in a sentence by its effect on entailment relations with another sentence. We show that a multilabel NLI model can flag political claims in the wild that are misleading due to ambiguity.
  arXiv  Detail & Related papers  (2023-04-27T17:57:58Z)
• Challenges in Automated Debiasing for Toxic Language Detection [81.04406231100323]
  Biased associations have been a challenge in the development of classifiers for detecting toxic language. We investigate recently introduced debiasing methods for text classification datasets and models, as applied to toxic language detection. Our focus is on lexical (e.g., swear words, slurs, identity mentions) and dialectal markers (specifically African American English)
  arXiv  Detail & Related papers  (2021-01-29T22:03:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.