Computational modeling of early language learning from acoustic speech and audiovisual input without linguistic priors

• URL: http://arxiv.org/abs/2603.08359v2
• Date: Wed, 11 Mar 2026 07:42:45 GMT
• Title: Computational modeling of early language learning from acoustic speech and audiovisual input without linguistic priors
• Abstract summary: This chapter reviews recent developments in using computational models to understand early language acquisition from speech and audiovisual input.<n>We show how these models are becoming increasingly powerful in learning various aspects of speech without strong linguistic priors.<n>We also discuss how modern learning simulations are gradually becoming more realistic, both in terms of input data and in linking model behavior to empirical findings on infant language development.
• Score: 6.572270548318532
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning to understand speech appears almost effortless for typically developing infants, yet from an information-processing perspective, acquiring a language from acoustic speech is an enormous challenge. This chapter reviews recent developments in using computational models to understand early language acquisition from speech and audiovisual input. The focus is on self-supervised and visually grounded models of perceptual learning. We show how these models are becoming increasingly powerful in learning various aspects of speech without strong linguistic priors, and how many features of early language development can be explained through a shared set of learning principles-principles broadly compatible with multiple theories of language acquisition and human cognition. We also discuss how modern learning simulations are gradually becoming more realistic, both in terms of input data and in linking model behavior to empirical findings on infant language development.

Related papers

• Towards Developmentally Plausible Rewards: Communicative Success as a Learning Signal for Interactive Language Models [49.22720751953838]
  We propose a method for training language models in an interactive setting inspired by child language acquisition.<n>In our setting, a speaker attempts to communicate some information to a listener in a single-turn dialogue and receives a reward if communicative success is achieved.
  arXiv  Detail & Related papers  (2025-05-09T11:48:36Z)
• OpenOmni: Advancing Open-Source Omnimodal Large Language Models with Progressive Multimodal Alignment and Real-Time Self-Aware Emotional Speech Synthesis [95.27191872116306]
  name is a two-stage training framework that integrates omnimodal alignment and speech generation.<n>It surpasses state-of-the-art models across omnimodal, vision-language, and speech-language benchmarks.<n>name achieves real-time speech generation with 1s latency at non-autoregressive mode.
  arXiv  Detail & Related papers  (2025-01-08T15:18:09Z)
• Developmental Predictive Coding Model for Early Infancy Mono and Bilingual Vocal Continual Learning [69.8008228833895]
  We propose a small-sized generative neural network equipped with a continual learning mechanism.<n>Our model prioritizes interpretability and demonstrates the advantages of online learning.
  arXiv  Detail & Related papers  (2024-12-23T10:23:47Z)
• Visual Grounding Helps Learn Word Meanings in Low-Data Regimes [47.7950860342515]
  Modern neural language models (LMs) are powerful tools for modeling human sentence production and comprehension. But to achieve these results, LMs must be trained in distinctly un-human-like ways. Do models trained more naturalistically -- with grounded supervision -- exhibit more humanlike language learning? We investigate this question in the context of word learning, a key sub-task in language acquisition.
  arXiv  Detail & Related papers  (2023-10-20T03:33:36Z)
• BabySLM: language-acquisition-friendly benchmark of self-supervised spoken language models [56.93604813379634]
  Self-supervised techniques for learning speech representations have been shown to develop linguistic competence from exposure to speech without the need for human labels. We propose a language-acquisition-friendly benchmark to probe spoken language models at the lexical and syntactic levels. We highlight two exciting challenges that need to be addressed for further progress: bridging the gap between text and speech and between clean speech and in-the-wild speech.
  arXiv  Detail & Related papers  (2023-06-02T12:54:38Z)
• Communication Drives the Emergence of Language Universals in Neural Agents: Evidence from the Word-order/Case-marking Trade-off [3.631024220680066]
  We propose a new Neural-agent Language Learning and Communication framework (NeLLCom) where pairs of speaking and listening agents first learn a miniature language. We succeed in replicating the trade-off with the new framework without hard-coding specific biases in the agents.
  arXiv  Detail & Related papers  (2023-01-30T17:22:33Z)
• What Artificial Neural Networks Can Tell Us About Human Language Acquisition [47.761188531404066]
  Rapid progress in machine learning for natural language processing has the potential to transform debates about how humans learn language. To increase the relevance of learnability results from computational models, we need to train model learners without significant advantages over humans.
  arXiv  Detail & Related papers  (2022-08-17T00:12:37Z)
• Bridging the Gap: Using Deep Acoustic Representations to Learn Grounded Language from Percepts and Raw Speech [26.076534338576234]
  Learning to understand grounded language, which connects natural language to percepts, is a critical research area. In this work we demonstrate the feasibility of performing grounded language acquisition on paired visual percepts and raw speech inputs.
  arXiv  Detail & Related papers  (2021-12-27T16:12:30Z)
• Perception Point: Identifying Critical Learning Periods in Speech for Bilingual Networks [58.24134321728942]
  We compare and identify cognitive aspects on deep neural-based visual lip-reading models. We observe a strong correlation between these theories in cognitive psychology and our unique modeling.
  arXiv  Detail & Related papers  (2021-10-13T05:30:50Z)
• Can phones, syllables, and words emerge as side-products of cross-situational audiovisual learning? -- A computational investigation [2.28438857884398]
  We study the so-called latent language hypothesis (LLH) LLH connects linguistic representation learning to general predictive processing within and across sensory modalities. We explore LLH further in extensive learning simulations with different neural network models for audiovisual cross-situational learning.
  arXiv  Detail & Related papers  (2021-09-29T05:49:46Z)
• Wav-BERT: Cooperative Acoustic and Linguistic Representation Learning for Low-Resource Speech Recognition [159.9312272042253]
  Wav-BERT is a cooperative acoustic and linguistic representation learning method. We unify a pre-trained acoustic model (wav2vec 2.0) and a language model (BERT) into an end-to-end trainable framework.
  arXiv  Detail & Related papers  (2021-09-19T16:39:22Z)
• Evaluating computational models of infant phonetic learning across languages [31.587496924289972]
  In the first year of life, infants' speech perception becomes attuned to the sounds of their native language. Many accounts of this early phonetic learning exist, but computational models predicting the patterns observed in infants from the speech input they hear have been lacking. Here we study five such algorithms, selected for their potential cognitive relevance. We simulate phonetic learning with each algorithm and perform tests on three phone contrasts from different languages, comparing the results to infants' discrimination patterns.
  arXiv  Detail & Related papers  (2020-08-06T22:07:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.