Implicit Representations of Grammaticality in Language Models

• URL: http://arxiv.org/abs/2605.05197v1
• Date: Wed, 06 May 2026 17:57:31 GMT
• Title: Implicit Representations of Grammaticality in Language Models
• Abstract summary: We train a linear probe on a dataset of grammatical and (synthetic) ungrammatical sentences.<n>We find that this simple grammaticality probe generalizes to human-curated grammaticality judgment benchmarks and outperforms LM probability-based grammaticality judgments.<n>The English-trained probe also exhibits nontrivial cross-lingual generalization, outperforming string probabilities on grammaticality benchmarks in numerous other languages.
• Score: 33.563784182750474
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Grammaticality and likelihood are distinct notions in human language. Pretrained language models (LMs), which are probabilistic models of language fitted to maximize corpus likelihood, generate grammatically well-formed text and discriminate well between grammatical and ungrammatical sentences in tightly controlled minimal pairs. However, their string probabilities do not sharply discriminate between grammatical and ungrammatical sentences overall. But do LMs implicitly acquire a grammaticality distinction distinct from string probability? We explore this question through studying internal representations of LMs, by training a linear probe on a dataset of grammatical and (synthetic) ungrammatical sentences obtained by applying perturbations to a naturalistic text corpus. We find that this simple grammaticality probe generalizes to human-curated grammaticality judgment benchmarks and outperforms LM probability-based grammaticality judgments. When applied to semantic plausibility benchmarks, in which both members of a minimal pair are grammatical and differ in only plausibility, the probe however performs worse than string probability. The English-trained probe also exhibits nontrivial cross-lingual generalization, outperforming string probabilities on grammaticality benchmarks in numerous other languages. Additionally, probe scores correlate only weakly with string probabilities. These results collectively suggest that LMs acquire to some extent an implicit grammaticality distinction within their hidden layers.

Related papers

• Linear representations of grammaticality in neural language models [7.771160777932259]
  We investigate whether grammaticality is encoded in the internal representations of neural language models.<n>Our results provide evidence that grammaticality is robustly encoded in sentence representations of a wide range of pretrained NLMs.
  arXiv  Detail & Related papers  (2026-07-16T16:23:56Z)
• What Can String Probability Tell Us About Grammaticality? [11.216210921392637]
  We present a theoretical analysis of the relationship between grammar, meaning, and string probability, based on simple assumptions about the generative process of corpus data.<n>Our framework makes three predictions, which we validate empirically using 280K sentence pairs in English and Chinese.<n>Our analyses give theoretical grounding for using probability to learn about LMs' structural knowledge, and suggest directions for future work in LM grammatical evaluation.
  arXiv  Detail & Related papers  (2025-10-17T21:36:00Z)
• Large Language Model probabilities cannot distinguish between possible and impossible language [0.11726720776908521]
  We use model-internal representations to tap directly into the way Large Language Models represent the 'grammatical-ungrammatical' distinction.<n>We predict that if string-probabilities can function as proxies for the limits of grammar, the ungrammatical condition will stand out among the conditions that involve linguistic violations.<n>Our results do not reveal a unique surprisal signature for ungrammatical prompts, as the semantically and pragmatically odd conditions consistently show higher surprisal.
  arXiv  Detail & Related papers  (2025-09-18T16:17:48Z)
• Inconsistent Tokenizations Cause Language Models to be Perplexed by Japanese Grammar [27.3347020320559]
  We measure the perplexity of language models when confronted with the "first person psych predicate restriction" grammar point in Japanese.<n>We show in further experiments that language models will use alternative grammar patterns in order to produce grammatical sentences when tokenization issues prevent the most natural sentence from being output.
  arXiv  Detail & Related papers  (2025-05-26T07:08:47Z)
• Can Language Models Learn Typologically Implausible Languages? [62.823015163987996]
  Grammatical features across human languages show intriguing correlations often attributed to learning biases in humans.<n>We discuss how language models (LMs) allow us to better determine the role of domain-general learning biases in language universals.<n>We test LMs on an array of highly naturalistic but counterfactual versions of the English (head-initial) and Japanese (head-final) languages.
  arXiv  Detail & Related papers  (2025-02-17T20:40:01Z)
• Large Language Models can Contrastively Refine their Generation for Better Sentence Representation Learning [57.74233319453229]
  Large language models (LLMs) have emerged as a groundbreaking technology and their unparalleled text generation capabilities have sparked interest in their application to the fundamental sentence representation learning task. We propose MultiCSR, a multi-level contrastive sentence representation learning framework that decomposes the process of prompting LLMs to generate a corpus. Our experiments reveal that MultiCSR enables a less advanced LLM to surpass the performance of ChatGPT, while applying it to ChatGPT achieves better state-of-the-art results.
  arXiv  Detail & Related papers  (2023-10-17T03:21:43Z)
• CLSE: Corpus of Linguistically Significant Entities [58.29901964387952]
  We release a Corpus of Linguistically Significant Entities (CLSE) annotated by experts. CLSE covers 74 different semantic types to support various applications from airline ticketing to video games. We create a linguistically representative NLG evaluation benchmark in three languages: French, Marathi, and Russian.
  arXiv  Detail & Related papers  (2022-11-04T12:56:12Z)
• Evaluating Distributional Distortion in Neural Language Modeling [81.83408583979745]
  A heavy-tail of rare events accounts for a significant amount of the total probability mass of distributions in language. Standard language modeling metrics such as perplexity quantify the performance of language models (LM) in aggregate. We develop a controlled evaluation scheme which uses generative models trained on natural data as artificial languages.
  arXiv  Detail & Related papers  (2022-03-24T01:09:46Z)
• Locally Typical Sampling [84.62530743899025]
  We show that today's probabilistic language generators fall short when it comes to producing coherent and fluent text.<n>We propose a simple and efficient procedure for enforcing this criterion when generating from probabilistic models.
  arXiv  Detail & Related papers  (2022-02-01T18:58:45Z)
• Self-Training Sampling with Monolingual Data Uncertainty for Neural Machine Translation [98.83925811122795]
  We propose to improve the sampling procedure by selecting the most informative monolingual sentences to complement the parallel data. We compute the uncertainty of monolingual sentences using the bilingual dictionary extracted from the parallel data. Experimental results on large-scale WMT English$Rightarrow$German and English$Rightarrow$Chinese datasets demonstrate the effectiveness of the proposed approach.
  arXiv  Detail & Related papers  (2021-06-02T05:01:36Z)
• Recurrent Neural Network Language Models Always Learn English-Like Relative Clause Attachment [17.995905582226463]
  We compare model performance in English and Spanish to show that non-linguistic biases in RNN LMs advantageously overlap with syntactic structure in English but not Spanish. English models may appear to acquire human-like syntactic preferences, while models trained on Spanish fail to acquire comparable human-like preferences.
  arXiv  Detail & Related papers  (2020-05-01T01:21:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.