Growing Trees on Sounds: Assessing Strategies for End-to-End Dependency Parsing of Speech

• URL: http://arxiv.org/abs/2406.12621v1
• Date: Tue, 18 Jun 2024 13:46:10 GMT
• Title: Growing Trees on Sounds: Assessing Strategies for End-to-End Dependency Parsing of Speech
• Authors: Adrien Pupier, Maximin Coavoux, Jérôme Goulian, Benjamin Lecouteux,
• Abstract summary: We report on a set of experiments aiming at assessing the performance of two parsing paradigms on speech parsing. We perform this evaluation on a large treebank of spoken French, featuring realistic spontaneous conversations. Our findings show that (i) the graph based approach obtains better results across the board (ii) parsing directly from speech outperforms a pipeline approach, despite having 30% fewer parameters.
• Score: 8.550564152063522
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Direct dependency parsing of the speech signal -- as opposed to parsing speech transcriptions -- has recently been proposed as a task (Pupier et al. 2022), as a way of incorporating prosodic information in the parsing system and bypassing the limitations of a pipeline approach that would consist of using first an Automatic Speech Recognition (ASR) system and then a syntactic parser. In this article, we report on a set of experiments aiming at assessing the performance of two parsing paradigms (graph-based parsing and sequence labeling based parsing) on speech parsing. We perform this evaluation on a large treebank of spoken French, featuring realistic spontaneous conversations. Our findings show that (i) the graph based approach obtain better results across the board (ii) parsing directly from speech outperforms a pipeline approach, despite having 30% fewer parameters.

Related papers

• Textless Dependency Parsing by Labeled Sequence Prediction [18.32371054754222]
  "textless" methods process speech representations without automatic speech recognition systems. Our proposed method predicts a dependency tree from a speech signal without transcribing, representing the tree as a labeled sequence. Our findings highlight the importance of fusing word-level representations and sentence-level prosody for enhanced parsing performance.
  arXiv  Detail & Related papers  (2024-07-14T08:38:14Z)
• Cascading and Direct Approaches to Unsupervised Constituency Parsing on Spoken Sentences [67.37544997614646]
  We present the first study on unsupervised spoken constituency parsing. The goal is to determine the spoken sentences' hierarchical syntactic structure in the form of constituency parse trees. We show that accurate segmentation alone may be sufficient to parse spoken sentences accurately.
  arXiv  Detail & Related papers  (2023-03-15T17:57:22Z)
• ESSumm: Extractive Speech Summarization from Untranscribed Meeting [7.309214379395552]
  We propose a novel architecture for direct extractive speech-to-speech summarization, ESSumm. We leverage the off-the-shelf self-supervised convolutional neural network to extract the deep speech features from raw audio. Our approach automatically predicts the optimal sequence of speech segments that capture the key information with a target summary length.
  arXiv  Detail & Related papers  (2022-09-14T20:13:15Z)
• Speaker Embedding-aware Neural Diarization: a Novel Framework for Overlapped Speech Diarization in the Meeting Scenario [51.5031673695118]
  We reformulate overlapped speech diarization as a single-label prediction problem. We propose the speaker embedding-aware neural diarization (SEND) system.
  arXiv  Detail & Related papers  (2022-03-18T06:40:39Z)
• Direct speech-to-speech translation with discrete units [64.19830539866072]
  We present a direct speech-to-speech translation (S2ST) model that translates speech from one language to speech in another language without relying on intermediate text generation. We propose to predict the self-supervised discrete representations learned from an unlabeled speech corpus instead. When target text transcripts are available, we design a multitask learning framework with joint speech and text training that enables the model to generate dual mode output (speech and text) simultaneously in the same inference pass.
  arXiv  Detail & Related papers  (2021-07-12T17:40:43Z)
• Leveraging Pre-trained Language Model for Speech Sentiment Analysis [58.78839114092951]
  We explore the use of pre-trained language models to learn sentiment information of written texts for speech sentiment analysis. We propose a pseudo label-based semi-supervised training strategy using a language model on an end-to-end speech sentiment approach.
  arXiv  Detail & Related papers  (2021-06-11T20:15:21Z)
• RST Parsing from Scratch [14.548146390081778]
  We introduce a novel end-to-end formulation of document-level discourse parsing in the Rhetorical Structure Theory (RST) framework. Our framework facilitates discourse parsing from scratch without requiring discourse segmentation as a prerequisite. Our unified parsing model adopts a beam search to decode the best tree structure by searching through a space of high-scoring trees.
  arXiv  Detail & Related papers  (2021-05-23T06:19:38Z)
• Syntactic representation learning for neural network based TTS with syntactic parse tree traversal [49.05471750563229]
  We propose a syntactic representation learning method based on syntactic parse tree to automatically utilize the syntactic structure information. Experimental results demonstrate the effectiveness of our proposed approach. For sentences with multiple syntactic parse trees, prosodic differences can be clearly perceived from the synthesized speeches.
  arXiv  Detail & Related papers  (2020-12-13T05:52:07Z)
• MEGA RST Discourse Treebanks with Structure and Nuclearity from Scalable Distant Sentiment Supervision [30.615883375573432]
  We present a novel methodology to automatically generate discourse treebanks using distant supervision from sentiment-annotated datasets. Our approach generates trees incorporating structure and nuclearity for documents of arbitrary length by relying on an efficient beam-search strategy. Experiments indicate that a discourse trained on our MEGA-DT treebank delivers promising inter-domain performance gains.
  arXiv  Detail & Related papers  (2020-11-05T18:22:38Z)
• Continuous speech separation: dataset and analysis [52.10378896407332]
  In natural conversations, a speech signal is continuous, containing both overlapped and overlap-free components. This paper describes a dataset and protocols for evaluating continuous speech separation algorithms.
  arXiv  Detail & Related papers  (2020-01-30T18:01:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.