Rethinking the Idiomaticity Decomposability Hypothesis: Evidence from Distributional Learning

• URL: http://arxiv.org/abs/2606.03817v1
• Date: Tue, 02 Jun 2026 15:59:22 GMT
• Title: Rethinking the Idiomaticity Decomposability Hypothesis: Evidence from Distributional Learning
• Abstract summary: We propose a model-internal measure of decomposability and relate it to human ratings, syntactic flexibility, and predictability while tracking learning during pretraining.<n>We show that stabilisation of representations in models is not explained by frequency alone. Instead surprisal decomposability, and frequency all contribute with decomposability showing the strongest training-dependent effect.
• Score: 15.730654942954267
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Idioms can be analysed in terms of their decomposability, the extent to which constituent meanings contribute to the figurative whole. Decomposability is thought to predict syntactic flexibility. Usage-based accounts instead attribute idiom behaviour to distributional experience, such as speaker familiarity and predictability. We examine these views using contextualised language models as controlled distributional learners. We propose a model-internal measure of decomposability and relate it to human ratings, syntactic flexibility, and predictability while tracking idiom learning during pretraining. Model-derived decomposability correlates weakly with human judgments and shows a small but consistent negative relationship with syntactic flexibility. Pretraining analyses show that stabilisation of idiom representations in models is not explained by frequency alone. Instead, surprisal, decomposability, and frequency all contribute, with decomposability showing the strongest training-dependent effect.

Related papers

• How Unlikely Is "Unlikely"? Assessing Verbal Probability Perception Across Large Language Models [1.435579512127234]
  We present a systematic cross-model evaluation using a word-to-number mapping task grounded in established human benchmarks.<n>Eleven uncertainty expressions were presented to 19 models under two conditions, forced single-number response and explanation elicitation.<n>Models show a systematic upward bias for negative expressions such as unlikely'' and improbable''
  arXiv  Detail & Related papers  (2026-08-26T19:00:41Z)
• Causally Evaluating the Learnability of Formal Language Tasks [59.73001113068827]
  We rigorously investigate the relationship between data frequency and learnability.<n>We derive Kullback-Leibler divergence metrics to measure the learnability of specific sub-tasks.<n>Our experiments show that evaluating learnability without causal intervention leads to incorrect conclusions.
  arXiv  Detail & Related papers  (2026-06-08T17:58:36Z)
• Do Language Models Know What Not to Say? Causal Evidence for Statistical Preemption in LLMs [13.891522069967507]
  Construction Grammar proposes statistical preemption: exposure to a conventional form preempts structurally possible but unattested alternatives.<n>We present a computational study that dissociates statistical preemption from the competing entrenchment hypothesis in large language models.
  arXiv  Detail & Related papers  (2026-05-21T21:06:43Z)
• Statistical Inference via Generative Models: Flow Matching and Causal Inference [0.0]
  Generative AI has achieved remarkable empirical success, but from the perspective of statistics it often remains opaque.<n>This book reinterprets generative AI in the language of statistics, using flow matching as a central example.
  arXiv  Detail & Related papers  (2026-03-09T22:56:02Z)
• Linguistic Indicators of Early Cognitive Decline in the DementiaBank Pitt Corpus: A Statistical and Machine Learning Study [4.417564179511245]
  This study analyzes spontaneous speech transcripts from the DementiaBank Pitt Corpus using three linguistic representations.<n> syntactic and grammatical features retain strong discriminative power even in the absence of lexical content.<n>This study supports the use of linguistically grounded features for transparent and reliable language-based cognitive screening.
  arXiv  Detail & Related papers  (2026-02-11T16:53:57Z)
• Enhancing Pre-trained Representation Classifiability can Boost its Interpretability [112.296393156262]
  We quantify the representation interpretability by leveraging its correlation with the ratio of interpretable semantics within the representations.<n>We propose the Inherent Interpretability Score (IIS) that evaluates the information loss, measures the ratio of interpretable semantics, and quantifies the representation interpretability.
  arXiv  Detail & Related papers  (2025-10-28T06:21:06Z)
• Readability $\ne$ Learnability: Rethinking the Role of Simplicity in Training Small Language Models [33.13548175654642]
  Recent studies suggest that very small language models (SLMs) can generate surprisingly coherent text when trained on child-directed corpora such as TinyStories.<n>These findings have been interpreted as evidence that readability plays a key role in enabling such capabilities to emerge.<n>We construct synthetic datasets with matched structure but varied readability, and find that readability alone does not predict coherence or learning efficiency in SLMs.
  arXiv  Detail & Related papers  (2025-10-15T08:17:02Z)
• Counterfactual reasoning: an analysis of in-context emergence [57.118735341305786]
  We show that language models are capable of counterfactual reasoning.<n>We find that self-attention, model depth and pre-training data diversity drive performance.<n>Our findings extend to counterfactual reasoning under SDE dynamics.
  arXiv  Detail & Related papers  (2025-06-05T16:02:07Z)
• Toward Understanding In-context vs. In-weight Learning [50.24035812301655]
  We identify simplified distributional properties that give rise to the emergence and disappearance of in-context learning.<n>We then extend the study to a full large language model, showing how fine-tuning on various collections of natural language prompts can elicit similar in-context and in-weight learning behaviour.
  arXiv  Detail & Related papers  (2024-10-30T14:09:00Z)
• Beyond Interpretability: The Gains of Feature Monosemanticity on Model Robustness [68.69369585600698]
  Deep learning models often suffer from a lack of interpretability due to polysemanticity. Recent advances in monosemanticity, where neurons correspond to consistent and distinct semantics, have significantly improved interpretability. We show that monosemantic features not only enhance interpretability but also bring concrete gains in model performance.
  arXiv  Detail & Related papers  (2024-10-27T18:03:20Z)
• Improving Factual Consistency Between a Response and Persona Facts [64.30785349238619]
  Neural models for response generation produce responses that are semantically plausible but not necessarily factually consistent with facts describing the speaker's persona. We propose to fine-tune these models by reinforcement learning and an efficient reward function that explicitly captures the consistency between a response and persona facts as well as semantic plausibility.
  arXiv  Detail & Related papers  (2020-04-30T18:08:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.