Information-Restricted Neural Language Models Reveal Different Brain Regions' Sensitivity to Semantics, Syntax and Context

• URL: http://arxiv.org/abs/2302.14389v1
• Date: Tue, 28 Feb 2023 08:16:18 GMT
• Title: Information-Restricted Neural Language Models Reveal Different Brain Regions' Sensitivity to Semantics, Syntax and Context
• Authors: Alexandre Pasquiou, Yair Lakretz, Bertrand Thirion, Christophe Pallier
• Abstract summary: We trained a lexical language model, Glove, and a supra-lexical language model, GPT-2, on a text corpus. We then assessed to what extent these information-restricted models were able to predict the time-courses of fMRI signal of humans listening to naturalistic text. Our analyses show that, while most brain regions involved in language are sensitive to both syntactic and semantic variables, the relative magnitudes of these effects vary a lot across these regions.
• Score: 87.31930367845125
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A fundamental question in neurolinguistics concerns the brain regions involved in syntactic and semantic processing during speech comprehension, both at the lexical (word processing) and supra-lexical levels (sentence and discourse processing). To what extent are these regions separated or intertwined? To address this question, we trained a lexical language model, Glove, and a supra-lexical language model, GPT-2, on a text corpus from which we selectively removed either syntactic or semantic information. We then assessed to what extent these information-restricted models were able to predict the time-courses of fMRI signal of humans listening to naturalistic text. We also manipulated the size of contextual information provided to GPT-2 in order to determine the windows of integration of brain regions involved in supra-lexical processing. Our analyses show that, while most brain regions involved in language are sensitive to both syntactic and semantic variables, the relative magnitudes of these effects vary a lot across these regions. Furthermore, we found an asymmetry between the left and right hemispheres, with semantic and syntactic processing being more dissociated in the left hemisphere than in the right, and the left and right hemispheres showing respectively greater sensitivity to short and long contexts. The use of information-restricted NLP models thus shed new light on the spatial organization of syntactic processing, semantic processing and compositionality.

Related papers

• Large Language Models as Neurolinguistic Subjects: Identifying Internal Representations for Form and Meaning [49.60849499134362]
  This study investigates the linguistic understanding of Large Language Models (LLMs) regarding signifier (form) and signified (meaning) Traditional psycholinguistic evaluations often reflect statistical biases that may misrepresent LLMs' true linguistic capabilities. We introduce a neurolinguistic approach, utilizing a novel method that combines minimal pair and diagnostic probing to analyze activation patterns across model layers.
  arXiv  Detail & Related papers  (2024-11-12T04:16:44Z)
• Unveiling A Core Linguistic Region in Large Language Models [49.860260050718516]
  This paper conducts an analogical research using brain localization as a prototype. We have discovered a core region in large language models that corresponds to linguistic competence. We observe that an improvement in linguistic competence does not necessarily accompany an elevation in the model's knowledge level.
  arXiv  Detail & Related papers  (2023-10-23T13:31:32Z)
• Do self-supervised speech and language models extract similar representations as human brain? [2.390915090736061]
  Speech and language models trained through self-supervised learning (SSL) demonstrate strong alignment with brain activity during speech and language perception. We evaluate the brain prediction performance of two representative SSL models, Wav2Vec2.0 and GPT-2.
  arXiv  Detail & Related papers  (2023-10-07T01:39:56Z)
• Probing Brain Context-Sensitivity with Masked-Attention Generation [87.31930367845125]
  We use GPT-2 transformers to generate word embeddings that capture a fixed amount of contextual information. We then tested whether these embeddings could predict fMRI brain activity in humans listening to naturalistic text.
  arXiv  Detail & Related papers  (2023-05-23T09:36:21Z)
• Joint processing of linguistic properties in brains and language models [14.997785690790032]
  We investigate the correspondence between the detailed processing of linguistic information by the human brain versus language models. We find that elimination of specific linguistic properties results in a significant decrease in brain alignment. These findings provide clear evidence for the role of specific linguistic information in the alignment between brain and language models.
  arXiv  Detail & Related papers  (2022-12-15T19:13:42Z)
• Low-Dimensional Structure in the Space of Language Representations is Reflected in Brain Responses [62.197912623223964]
  We show a low-dimensional structure where language models and translation models smoothly interpolate between word embeddings, syntactic and semantic tasks, and future word embeddings. We find that this representation embedding can predict how well each individual feature space maps to human brain responses to natural language stimuli recorded using fMRI. This suggests that the embedding captures some part of the brain's natural language representation structure.
  arXiv  Detail & Related papers  (2021-06-09T22:59:12Z)
• Decomposing lexical and compositional syntax and semantics with deep language models [82.81964713263483]
  The activations of language transformers like GPT2 have been shown to linearly map onto brain activity during speech comprehension. Here, we propose a taxonomy to factorize the high-dimensional activations of language models into four classes: lexical, compositional, syntactic, and semantic representations. The results highlight two findings. First, compositional representations recruit a more widespread cortical network than lexical ones, and encompass the bilateral temporal, parietal and prefrontal cortices.
  arXiv  Detail & Related papers  (2021-03-02T10:24:05Z)
• Does injecting linguistic structure into language models lead to better alignment with brain recordings? [13.880819301385854]
  We evaluate whether language models align better with brain recordings if their attention is biased by annotations from syntactic or semantic formalisms. Our proposed approach enables the evaluation of more targeted hypotheses about the composition of meaning in the brain.
  arXiv  Detail & Related papers  (2021-01-29T14:42:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.