Layer-wise Probing of wav2vec 2.0 and Whisper for Consonant Cluster Reduction in African American English

• URL: http://arxiv.org/abs/2606.23948v1
• Date: Mon, 22 Jun 2026 21:19:03 GMT
• Title: Layer-wise Probing of wav2vec 2.0 and Whisper for Consonant Cluster Reduction in African American English
• Abstract summary: Consonant cluster reduction ( CCR) is a widespread phonological process and a source of automatic speech recognition (ASR) disparity.<n>We show that reduced segments retain cues to their underlying stops, indicating that CCR is encoded as structured phonological variation.<n>These results demonstrate structured phonological encoding of AAE CCR patterns in modern speech models.
• Score: 6.164342356356261
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Self-supervised and supervised speech models are increasingly used to investigate which linguistic information their internal representations encode, and at what level of abstraction they encode it. One underexplored phenomenon is consonant cluster reduction (CCR) in African American English (AAE), a widespread phonological process and a source of automatic speech recognition (ASR) disparity. To examine how CCR is represented, we conduct speaker-independent layer-wise probing of wav2vec2-base and Whisper-small using two tasks: segmental reduction detection and segmental restoration of underlying cluster identity. Both models distinguish reduced and canonical forms with high accuracy. Crucially, reduced segments retain cues to their underlying stops, indicating that CCR is encoded as structured gradient phonological variation rather than simple segmental deletion. These results demonstrate structured phonological encoding of AAE CCR patterns in modern speech models.

Related papers

• WavCube: Unifying Speech Representation for Understanding and Generation via Semantic-Acoustic Joint Modeling [35.33131758542107]
  WavCube is a compact continuous latent derived from an SSL speech encoder.<n>It simultaneously supports speech understanding, reconstruction, and generation.<n>Trials show it delivers state-of-the-art zero-shot TTS performance with markedly faster training convergence.
  arXiv  Detail & Related papers  (2026-05-07T15:17:24Z)
• Factorized RVQ-GAN For Disentangled Speech Tokenization [34.73812744103814]
  We propose a unified neural speech encoder that factorizes its bottleneck into three linguistic levels-acoustic, phonetic, and lexical-within a single model.<n> Experiments on English and multilingual data show that HAC's factorized bottleneck yields disentangled token sets.<n> Quantitative evaluations confirm that HAC preserve naturalness and provide interpretable linguistic information, outperforming single-level baselines in both disentanglement and reconstruction quality.
  arXiv  Detail & Related papers  (2025-06-18T13:36:34Z)
• Contrastive and Consistency Learning for Neural Noisy-Channel Model in Spoken Language Understanding [1.07288078404291]
  We propose a natural language understanding approach based on Automatic Speech Recognition (ASR) We improve a noisy-channel model to handle transcription inconsistencies caused by ASR errors. Experiments on four benchmark datasets show that Contrastive and Consistency Learning (CCL) outperforms existing methods.
  arXiv  Detail & Related papers  (2024-05-23T23:10:23Z)
• Speech collage: code-switched audio generation by collaging monolingual corpora [50.356820349870986]
  Speech Collage is a method that synthesizes CS data from monolingual corpora by splicing audio segments. We investigate the impact of generated data on speech recognition in two scenarios.
  arXiv  Detail & Related papers  (2023-09-27T14:17:53Z)
• High-Fidelity Speech Synthesis with Minimal Supervision: All Using Diffusion Models [56.00939852727501]
  Minimally-supervised speech synthesis decouples TTS by combining two types of discrete speech representations. Non-autoregressive framework enhances controllability, and duration diffusion model enables diversified prosodic expression.
  arXiv  Detail & Related papers  (2023-09-27T09:27:03Z)
• Self-supervised models of audio effectively explain human cortical responses to speech [71.57870452667369]
  We capitalize on the progress of self-supervised speech representation learning to create new state-of-the-art models of the human auditory system. We show that these results show that self-supervised models effectively capture the hierarchy of information relevant to different stages of speech processing in human cortex.
  arXiv  Detail & Related papers  (2022-05-27T22:04:02Z)
• Bi-LSTM Scoring Based Similarity Measurement with Agglomerative Hierarchical Clustering (AHC) for Speaker Diarization [0.0]
  A typical conversation between two speakers consists of segments where their voices overlap, interrupt each other or halt their speech in between multiple sentences. Recent advancements in diarization technology leverage neural network-based approaches to improvise speaker diarization system. We propose a Bi-directional Long Short-term Memory network for estimating the elements present in the similarity matrix.
  arXiv  Detail & Related papers  (2022-05-19T17:20:51Z)
• Chain-based Discriminative Autoencoders for Speech Recognition [16.21321835306968]
  We propose three new versions of a discriminative autoencoder (DcAE) for speech recognition. First, a new objective function that considers both categorical cross-entropy and mutual information between ground truth and predicted triphone-state sequences is used. For application to robust speech recognition, we extend c-DcAE to hierarchical and parallel structures, resulting in hc-DcAE and pc-DcAE.
  arXiv  Detail & Related papers  (2022-03-25T14:51:48Z)
• Speaker Embedding-aware Neural Diarization: a Novel Framework for Overlapped Speech Diarization in the Meeting Scenario [51.5031673695118]
  We reformulate overlapped speech diarization as a single-label prediction problem. We propose the speaker embedding-aware neural diarization (SEND) system.
  arXiv  Detail & Related papers  (2022-03-18T06:40:39Z)
• FragmentVC: Any-to-Any Voice Conversion by End-to-End Extracting and Fusing Fine-Grained Voice Fragments With Attention [66.77490220410249]
  We propose FragmentVC, in which the latent phonetic structure of the utterance from the source speaker is obtained from Wav2Vec 2.0. FragmentVC is able to extract fine-grained voice fragments from the target speaker utterance(s) and fuse them into the desired utterance. This approach is trained with reconstruction loss only without any disentanglement considerations between content and speaker information.
  arXiv  Detail & Related papers  (2020-10-27T09:21:03Z)
• Audio-visual Multi-channel Recognition of Overlapped Speech [79.21950701506732]
  This paper presents an audio-visual multi-channel overlapped speech recognition system featuring tightly integrated separation front-end and recognition back-end. Experiments suggest that the proposed multi-channel AVSR system outperforms the baseline audio-only ASR system by up to 6.81% (26.83% relative) and 22.22% (56.87% relative) absolute word error rate (WER) reduction on overlapped speech constructed using either simulation or replaying of the lipreading sentence 2 dataset respectively.
  arXiv  Detail & Related papers  (2020-05-18T10:31:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.