Learning to Understand Child-directed and Adult-directed Speech

• URL: http://arxiv.org/abs/2005.02721v4
• Date: Fri, 16 Jul 2021 15:25:26 GMT
• Title: Learning to Understand Child-directed and Adult-directed Speech
• Authors: Lieke Gelderloos, Grzegorz Chrupa{\l}a, Afra Alishahi
• Abstract summary: Human language acquisition research indicates that child-directed speech helps language learners. We compare the task performance of models trained on adult-directed speech (ADS) and child-directed speech (CDS) We find indications that CDS helps in the initial stages of learning, but eventually, models trained on ADS reach comparable task performance, and generalize better.
• Score: 18.29692441616062
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Speech directed to children differs from adult-directed speech in linguistic aspects such as repetition, word choice, and sentence length, as well as in aspects of the speech signal itself, such as prosodic and phonemic variation. Human language acquisition research indicates that child-directed speech helps language learners. This study explores the effect of child-directed speech when learning to extract semantic information from speech directly. We compare the task performance of models trained on adult-directed speech (ADS) and child-directed speech (CDS). We find indications that CDS helps in the initial stages of learning, but eventually, models trained on ADS reach comparable task performance, and generalize better. The results suggest that this is at least partially due to linguistic rather than acoustic properties of the two registers, as we see the same pattern when looking at models trained on acoustically comparable synthetic speech.

Related papers

• Cross-lingual Speech Emotion Recognition: Humans vs. Self-Supervised Models [16.0617753653454]
  This study presents a comparative analysis between human performance and SSL models. We also compare the SER ability of models and humans at both utterance- and segment-levels. Our findings reveal that models, with appropriate knowledge transfer, can adapt to the target language and achieve performance comparable to native speakers.
  arXiv  Detail & Related papers  (2024-09-25T13:27:17Z)
• Speech2rtMRI: Speech-Guided Diffusion Model for Real-time MRI Video of the Vocal Tract during Speech [29.510756530126837]
  We introduce a data-driven method to visually represent articulator motion in MRI videos of the human vocal tract during speech. We leverage large pre-trained speech models, which are embedded with prior knowledge, to generalize the visual domain to unseen data.
  arXiv  Detail & Related papers  (2024-09-23T20:19:24Z)
• Improved Child Text-to-Speech Synthesis through Fastpitch-based Transfer Learning [3.5032870024762386]
  This paper presents a novel approach that leverages the Fastpitch text-to-speech (TTS) model for generating high-quality synthetic child speech. The approach involved finetuning a multi-speaker TTS model to work with child speech. We conducted an objective assessment that showed a significant correlation between real and synthetic child voices.
  arXiv  Detail & Related papers  (2023-11-07T19:31:44Z)
• Audio-Visual Neural Syntax Acquisition [91.14892278795892]
  We study phrase structure induction from visually-grounded speech. We present the Audio-Visual Neural Syntax Learner (AV-NSL) that learns phrase structure by listening to audio and looking at images, without ever being exposed to text.
  arXiv  Detail & Related papers  (2023-10-11T16:54:57Z)
• Do self-supervised speech and language models extract similar representations as human brain? [2.390915090736061]
  Speech and language models trained through self-supervised learning (SSL) demonstrate strong alignment with brain activity during speech and language perception. We evaluate the brain prediction performance of two representative SSL models, Wav2Vec2.0 and GPT-2.
  arXiv  Detail & Related papers  (2023-10-07T01:39:56Z)
• Improving Children's Speech Recognition by Fine-tuning Self-supervised Adult Speech Representations [2.2191297646252646]
  Children's speech recognition is a vital, yet largely overlooked domain when building inclusive speech technologies. Recent advances in self-supervised learning have created a new opportunity for overcoming this problem of data scarcity. We leverage self-supervised adult speech representations and use three well-known child speech corpora to build models for children's speech recognition.
  arXiv  Detail & Related papers  (2022-11-14T22:03:36Z)
• M-SpeechCLIP: Leveraging Large-Scale, Pre-Trained Models for Multilingual Speech to Image Retrieval [56.49878599920353]
  This work investigates the use of large-scale, English-only pre-trained models (CLIP and HuBERT) for multilingual image-speech retrieval. For non-English image-speech retrieval, we outperform the current state-of-the-art performance by a wide margin both when training separate models for each language, and with a single model which processes speech in all three languages.
  arXiv  Detail & Related papers  (2022-11-02T14:54:45Z)
• Audio-Visual Speech Codecs: Rethinking Audio-Visual Speech Enhancement by Re-Synthesis [67.73554826428762]
  We propose a novel audio-visual speech enhancement framework for high-fidelity telecommunications in AR/VR. Our approach leverages audio-visual speech cues to generate the codes of a neural speech, enabling efficient synthesis of clean, realistic speech from noisy signals.
  arXiv  Detail & Related papers  (2022-03-31T17:57:10Z)
• Perception Point: Identifying Critical Learning Periods in Speech for Bilingual Networks [58.24134321728942]
  We compare and identify cognitive aspects on deep neural-based visual lip-reading models. We observe a strong correlation between these theories in cognitive psychology and our unique modeling.
  arXiv  Detail & Related papers  (2021-10-13T05:30:50Z)
• Mandarin-English Code-switching Speech Recognition with Self-supervised Speech Representation Models [55.82292352607321]
  Code-switching (CS) is common in daily conversations where more than one language is used within a sentence. This paper uses the recently successful self-supervised learning (SSL) methods to leverage many unlabeled speech data without CS.
  arXiv  Detail & Related papers  (2021-10-07T14:43:35Z)
• SPLAT: Speech-Language Joint Pre-Training for Spoken Language Understanding [61.02342238771685]
  Spoken language understanding requires a model to analyze input acoustic signal to understand its linguistic content and make predictions. Various pre-training methods have been proposed to learn rich representations from large-scale unannotated speech and text. We propose a novel semi-supervised learning framework, SPLAT, to jointly pre-train the speech and language modules.
  arXiv  Detail & Related papers  (2020-10-05T19:29:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.