The Holistic Storage of Verb+Up Phrases in Text-based and Audio-based Language Models

• URL: http://arxiv.org/abs/2606.13993v1
• Date: Fri, 12 Jun 2026 00:27:42 GMT
• Title: The Holistic Storage of Verb+Up Phrases in Text-based and Audio-based Language Models
• Abstract summary: holistic storage is driven by frequency and predictability.<n>All models show evidence of holistic storage driven by frequency and predictability, further supporting usage-based theories of language.
• Score: 2.835219195516112
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A crucial aspect of linguistic capability is the ability to trade off between stored representations and abstract knowledge: one must retrieve learned representations, but also generate novel ones by applying productive rules. While recent work has examined abstract knowledge in language models, holistic storage of multi-word units has received far less attention. We probe internal representations in text-based LLMs and an ASR model, testing whether V+up phrasal verbs develop distinct representations as a function of frequency and predictability. All models show evidence of holistic storage driven by frequency and predictability, further supporting usage-based theories of language.

Related papers

• Verbalizable Representations Form a Global Workspace in Language Models [13.088673968096884]
  We present evidence that an analogous functional distinction has emerged in large language models.<n>We identify the representations a model is poised to verbalize at any point in its processing.<n>We find that language models maintain a small, privileged set of representations bearing some of the functional hallmarks of conscious access.
  arXiv  Detail & Related papers  (2026-07-16T22:54:30Z)
• Shaping Schema via Language Representation as the Next Frontier for LLM Intelligence Expanding [88.11781604392606]
  This paper defines language representation as the linguistic and symbolic constructs used to map and model the real world.<n>We argue that shaping schemas through advanced language representation is the next frontier for expanding Large Language Models.
  arXiv  Detail & Related papers  (2026-05-10T02:42:29Z)
• Seeing the Abstract: Translating the Abstract Language for Vision Language Models [13.065703240655973]
  We focus our investigation on the fashion domain, a highly-representative field with abstract expressions.<n>By analyzing recent large-scale multimodal fashion datasets, we find that abstract terms have a dominant presence.<n>We propose a training-free and model-agnostic method, Abstract-to-Concrete Translator (ACT), to shift abstract representations towards well-represented concrete ones.
  arXiv  Detail & Related papers  (2025-05-06T07:14:10Z)
• LinguaLens: Towards Interpreting Linguistic Mechanisms of Large Language Models via Sparse Auto-Encoder [47.81850176849213]
  We propose a framework for analyzing the linguistic mechanisms of large language models, based on Sparse Auto-Encoders (SAEs)<n>We extract a broad set of Chinese and English linguistic features across four dimensions (morphology, syntax, semantics, and pragmatics)<n>Our findings reveal intrinsic representations of linguistic knowledge in LLMs, uncover patterns of cross-layer and cross-lingual distribution, and demonstrate the potential to control model outputs.
  arXiv  Detail & Related papers  (2025-02-27T18:16:47Z)
• AbsPyramid: Benchmarking the Abstraction Ability of Language Models with a Unified Entailment Graph [62.685920585838616]
  abstraction ability is essential in human intelligence, which remains under-explored in language models. We present AbsPyramid, a unified entailment graph of 221K textual descriptions of abstraction knowledge.
  arXiv  Detail & Related papers  (2023-11-15T18:11:23Z)
• Self-Supervised Speech Representation Learning: A Review [105.1545308184483]
  Self-supervised representation learning methods promise a single universal model that would benefit a wide variety of tasks and domains. Speech representation learning is experiencing similar progress in three main categories: generative, contrastive, and predictive methods. This review presents approaches for self-supervised speech representation learning and their connection to other research areas.
  arXiv  Detail & Related papers  (2022-05-21T16:52:57Z)
• Towards Language Modelling in the Speech Domain Using Sub-word Linguistic Units [56.52704348773307]
  We propose a novel LSTM-based generative speech LM based on linguistic units including syllables and phonemes. With a limited dataset, orders of magnitude smaller than that required by contemporary generative models, our model closely approximates babbling speech. We show the effect of training with auxiliary text LMs, multitask learning objectives, and auxiliary articulatory features.
  arXiv  Detail & Related papers  (2021-10-31T22:48:30Z)
• Can phones, syllables, and words emerge as side-products of cross-situational audiovisual learning? -- A computational investigation [2.28438857884398]
  We study the so-called latent language hypothesis (LLH) LLH connects linguistic representation learning to general predictive processing within and across sensory modalities. We explore LLH further in extensive learning simulations with different neural network models for audiovisual cross-situational learning.
  arXiv  Detail & Related papers  (2021-09-29T05:49:46Z)
• Wav-BERT: Cooperative Acoustic and Linguistic Representation Learning for Low-Resource Speech Recognition [159.9312272042253]
  Wav-BERT is a cooperative acoustic and linguistic representation learning method. We unify a pre-trained acoustic model (wav2vec 2.0) and a language model (BERT) into an end-to-end trainable framework.
  arXiv  Detail & Related papers  (2021-09-19T16:39:22Z)
• A learning perspective on the emergence of abstractions: the curious case of phonemes [2.580765958706854]
  We test two opposing principles regarding the development of language knowledge in linguistically untrained language users. We probed whether MBL and ECL could give rise to a type of language knowledge that resembles linguistic abstractions. We show that ECL learning models can learn abstractions and that at least part of the phone inventory can be reliably identified from the input.
  arXiv  Detail & Related papers  (2020-12-14T13:33:34Z)
• SPLAT: Speech-Language Joint Pre-Training for Spoken Language Understanding [61.02342238771685]
  Spoken language understanding requires a model to analyze input acoustic signal to understand its linguistic content and make predictions. Various pre-training methods have been proposed to learn rich representations from large-scale unannotated speech and text. We propose a novel semi-supervised learning framework, SPLAT, to jointly pre-train the speech and language modules.
  arXiv  Detail & Related papers  (2020-10-05T19:29:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.