A predictive learning model can simulate temporal dynamics and context effects found in neural representations of continuous speech

• URL: http://arxiv.org/abs/2405.08237v1
• Date: Mon, 13 May 2024 23:36:19 GMT
• Title: A predictive learning model can simulate temporal dynamics and context effects found in neural representations of continuous speech
• Authors: Oli Danyi Liu, Hao Tang, Naomi Feldman, Sharon Goldwater,
• Abstract summary: Recent work in cognitive neuroscience has identified temporal and contextual characteristics in humans' neural encoding of speech. In this study, we simulated similar analyses with representations extracted from a computational model that was trained on unlabelled speech. Our simulations revealed temporal dynamics similar to those in brain signals, implying that these properties can arise without linguistic knowledge.
• Score: 11.707968216076075
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Speech perception involves storing and integrating sequentially presented items. Recent work in cognitive neuroscience has identified temporal and contextual characteristics in humans' neural encoding of speech that may facilitate this temporal processing. In this study, we simulated similar analyses with representations extracted from a computational model that was trained on unlabelled speech with the learning objective of predicting upcoming acoustics. Our simulations revealed temporal dynamics similar to those in brain signals, implying that these properties can arise without linguistic knowledge. Another property shared between brains and the model is that the encoding patterns of phonemes support some degree of cross-context generalization. However, we found evidence that the effectiveness of these generalizations depends on the specific contexts, which suggests that this analysis alone is insufficient to support the presence of context-invariant encoding.

Related papers

• Discovering Chunks in Neural Embeddings for Interpretability [53.80157905839065]
  We propose leveraging the principle of chunking to interpret artificial neural population activities. We first demonstrate this concept in recurrent neural networks (RNNs) trained on artificial sequences with imposed regularities. We identify similar recurring embedding states corresponding to concepts in the input, with perturbations to these states activating or inhibiting the associated concepts.
  arXiv  Detail & Related papers  (2025-02-03T20:30:46Z)
• Modelling change in neural dynamics during phonetic accommodation [0.0]
  We advance a computational model of change in phonetic representations during phonetic accommodation. We show vowel-specific degrees of convergence during shadowing, followed by return to baseline post-shadowing. We discuss the implications for the relation between short-term phonetic accommodation and longer-term patterns of sound change.
  arXiv  Detail & Related papers  (2025-02-03T10:00:29Z)
• Analysis of Argument Structure Constructions in a Deep Recurrent Language Model [0.0]
  We explore the representation and processing of Argument Structure Constructions (ASCs) in a recurrent neural language model. Our results show that sentence representations form distinct clusters corresponding to the four ASCs across all hidden layers. This indicates that even a relatively simple, brain-constrained recurrent neural network can effectively differentiate between various construction types.
  arXiv  Detail & Related papers  (2024-08-06T09:27:41Z)
• Investigating the Timescales of Language Processing with EEG and Language Models [0.0]
  This study explores the temporal dynamics of language processing by examining the alignment between word representations from a pre-trained language model and EEG data. Using a Temporal Response Function (TRF) model, we investigate how neural activity corresponds to model representations across different layers. Our analysis reveals patterns in TRFs from distinct layers, highlighting varying contributions to lexical and compositional processing.
  arXiv  Detail & Related papers  (2024-06-28T12:49:27Z)
• Perception of Phonological Assimilation by Neural Speech Recognition Models [3.4173734484549625]
  This article explores how the neural speech recognition model Wav2Vec2 perceives assimilated sounds. Using psycholinguistic stimuli, we analyze how various linguistic context cues influence compensation patterns in the model's output.
  arXiv  Detail & Related papers  (2024-06-21T15:58:22Z)
• Exploring neural oscillations during speech perception via surrogate gradient spiking neural networks [59.38765771221084]
  We present a physiologically inspired speech recognition architecture compatible and scalable with deep learning frameworks. We show end-to-end gradient descent training leads to the emergence of neural oscillations in the central spiking neural network. Our findings highlight the crucial inhibitory role of feedback mechanisms, such as spike frequency adaptation and recurrent connections, in regulating and synchronising neural activity to improve recognition performance.
  arXiv  Detail & Related papers  (2024-04-22T09:40:07Z)
• The neural dynamics of auditory word recognition and integration [21.582292050622456]
  We present a computational model of word recognition which formalizes this perceptual process in Bayesian decision theory. We fit this model to explain scalp EEG signals recorded as subjects passively listened to a fictional story. The model reveals distinct neural processing of words depending on whether or not they can be quickly recognized.
  arXiv  Detail & Related papers  (2023-05-22T18:06:32Z)
• Deep Neural Convolutive Matrix Factorization for Articulatory Representation Decomposition [48.56414496900755]
  This work uses a neural implementation of convolutive sparse matrix factorization to decompose the articulatory data into interpretable gestures and gestural scores. Phoneme recognition experiments were additionally performed to show that gestural scores indeed code phonological information successfully.
  arXiv  Detail & Related papers  (2022-04-01T14:25:19Z)
• A Latent-Variable Model for Intrinsic Probing [93.62808331764072]
  We propose a novel latent-variable formulation for constructing intrinsic probes. We find empirical evidence that pre-trained representations develop a cross-lingually entangled notion of morphosyntax.
  arXiv  Detail & Related papers  (2022-01-20T15:01:12Z)
• Preliminary study on using vector quantization latent spaces for TTS/VC systems with consistent performance [55.10864476206503]
  We investigate the use of quantized vectors to model the latent linguistic embedding. By enforcing different policies over the latent spaces in the training, we are able to obtain a latent linguistic embedding. Our experiments show that the voice cloning system built with vector quantization has only a small degradation in terms of perceptive evaluations.
  arXiv  Detail & Related papers  (2021-06-25T07:51:35Z)
• Mechanisms for Handling Nested Dependencies in Neural-Network Language Models and Humans [75.15855405318855]
  We studied whether a modern artificial neural network trained with "deep learning" methods mimics a central aspect of human sentence processing. Although the network was solely trained to predict the next word in a large corpus, analysis showed the emergence of specialized units that successfully handled local and long-distance syntactic agreement. We tested the model's predictions in a behavioral experiment where humans detected violations in number agreement in sentences with systematic variations in the singular/plural status of multiple nouns.
  arXiv  Detail & Related papers  (2020-06-19T12:00:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.