Analyzing analytical methods: The case of phonology in neural models of spoken language

• URL: http://arxiv.org/abs/2004.07070v2
• Date: Sat, 2 May 2020 07:59:40 GMT
• Title: Analyzing analytical methods: The case of phonology in neural models of spoken language
• Authors: Grzegorz Chrupa{\l}a, Bertrand Higy, Afra Alishahi
• Abstract summary: We study the case of representations of phonology in neural network models of spoken language. We use two commonly applied analytical techniques to quantify to what extent neural activation patterns encode phonemes and phoneme sequences.
• Score: 44.00588930401902
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Given the fast development of analysis techniques for NLP and speech processing systems, few systematic studies have been conducted to compare the strengths and weaknesses of each method. As a step in this direction we study the case of representations of phonology in neural network models of spoken language. We use two commonly applied analytical techniques, diagnostic classifiers and representational similarity analysis, to quantify to what extent neural activation patterns encode phonemes and phoneme sequences. We manipulate two factors that can affect the outcome of analysis. First, we investigate the role of learning by comparing neural activations extracted from trained versus randomly-initialized models. Second, we examine the temporal scope of the activations by probing both local activations corresponding to a few milliseconds of the speech signal, and global activations pooled over the whole utterance. We conclude that reporting analysis results with randomly initialized models is crucial, and that global-scope methods tend to yield more consistent results and we recommend their use as a complement to local-scope diagnostic methods.

Related papers

• Investigating the Timescales of Language Processing with EEG and Language Models [0.0]
  This study explores the temporal dynamics of language processing by examining the alignment between word representations from a pre-trained language model and EEG data. Using a Temporal Response Function (TRF) model, we investigate how neural activity corresponds to model representations across different layers. Our analysis reveals patterns in TRFs from distinct layers, highlighting varying contributions to lexical and compositional processing.
  arXiv  Detail & Related papers  (2024-06-28T12:49:27Z)
• A predictive learning model can simulate temporal dynamics and context effects found in neural representations of continuous speech [11.707968216076075]
  Recent work in cognitive neuroscience has identified temporal and contextual characteristics in humans' neural encoding of speech. In this study, we simulated similar analyses with representations extracted from a computational model that was trained on unlabelled speech. Our simulations revealed temporal dynamics similar to those in brain signals, implying that these properties can arise without linguistic knowledge.
  arXiv  Detail & Related papers  (2024-05-13T23:36:19Z)
• Deep Neural Convolutive Matrix Factorization for Articulatory Representation Decomposition [48.56414496900755]
  This work uses a neural implementation of convolutive sparse matrix factorization to decompose the articulatory data into interpretable gestures and gestural scores. Phoneme recognition experiments were additionally performed to show that gestural scores indeed code phonological information successfully.
  arXiv  Detail & Related papers  (2022-04-01T14:25:19Z)
• Dependency-based Mixture Language Models [53.152011258252315]
  We introduce the Dependency-based Mixture Language Models. In detail, we first train neural language models with a novel dependency modeling objective. We then formulate the next-token probability by mixing the previous dependency modeling probability distributions with self-attention.
  arXiv  Detail & Related papers  (2022-03-19T06:28:30Z)
• Connecting Neural Response measurements & Computational Models of language: a non-comprehensive guide [5.523143941738335]
  Recent advances in language modelling and in neuroimaging promise potential improvements in the investigation of language's neurobiology. This survey traces a line from early research linking Event Related Potentials and complexity measures derived from simple language models to contemporary studies employing Artificial Neural Network models trained on large corpora.
  arXiv  Detail & Related papers  (2022-03-10T11:24:54Z)
• Model-based analysis of brain activity reveals the hierarchy of language in 305 subjects [82.81964713263483]
  A popular approach to decompose the neural bases of language consists in correlating, across individuals, the brain responses to different stimuli. Here, we show that a model-based approach can reach equivalent results within subjects exposed to natural stimuli.
  arXiv  Detail & Related papers  (2021-10-12T15:30:21Z)
• Exploiting Language Model for Efficient Linguistic Steganalysis: An Empirical Study [23.311007481830647]
  We present two methods to efficient linguistic steganalysis. One is to pre-train a language model based on RNN, and the other is to pre-train a sequence autoencoder.
  arXiv  Detail & Related papers  (2021-07-26T12:37:18Z)
• Discrete representations in neural models of spoken language [56.29049879393466]
  We compare the merits of four commonly used metrics in the context of weakly supervised models of spoken language. We find that the different evaluation metrics can give inconsistent results.
  arXiv  Detail & Related papers  (2021-05-12T11:02:02Z)
• Multi-modal Automated Speech Scoring using Attention Fusion [46.94442359735952]
  We propose a novel multi-modal end-to-end neural approach for automated assessment of non-native English speakers' spontaneous speech. We employ Bi-directional Recurrent Convolutional Neural Networks and Bi-directional Long Short-Term Memory Neural Networks to encode acoustic and lexical cues from spectrograms and transcriptions. We find combined attention to both lexical and acoustic cues significantly improves the overall performance of the system.
  arXiv  Detail & Related papers  (2020-05-17T07:53:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.