An Initial Investigation of Non-Native Spoken Question-Answering

• URL: http://arxiv.org/abs/2107.04691v1
• Date: Fri, 9 Jul 2021 21:59:16 GMT
• Title: An Initial Investigation of Non-Native Spoken Question-Answering
• Authors: Vatsal Raina, Mark J.F. Gales
• Abstract summary: We show that a simple text-based ELECTRA MC model trained on SQuAD2.0 transfers well for spoken question answering tests. One significant challenge is the lack of appropriately annotated speech corpora to train systems for this task. Mismatches must be considered between text documents and spoken responses; non-native spoken grammar and written grammar.
• Score: 36.89541375786233
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Text-based machine comprehension (MC) systems have a wide-range of applications, and standard corpora exist for developing and evaluating approaches. There has been far less research on spoken question answering (SQA) systems. The SQA task considered in this paper is to extract the answer from a candidate$\text{'}$s spoken response to a question in a prompt-response style language assessment test. Applying these MC approaches to this SQA task rather than, for example, off-topic response detection provides far more detailed information that can be used for further downstream processing. One significant challenge is the lack of appropriately annotated speech corpora to train systems for this task. Hence, a transfer-learning style approach is adopted where a system trained on text-based MC is evaluated on an SQA task with non-native speakers. Mismatches must be considered between text documents and spoken responses; non-native spoken grammar and written grammar. In practical SQA, ASR systems are used, necessitating an investigation of the impact of ASR errors. We show that a simple text-based ELECTRA MC model trained on SQuAD2.0 transfers well for SQA. It is found that there is an approximately linear relationship between ASR errors and the SQA assessment scores but grammar mismatches have minimal impact.

Related papers

• SpeechDPR: End-to-End Spoken Passage Retrieval for Open-Domain Spoken Question Answering [76.4510005602893]
  Spoken Question Answering (SQA) is essential for machines to reply to user's question by finding the answer span within a given spoken passage. This paper proposes the first known end-to-end framework, Speech Passage Retriever (SpeechDPR) SpeechDPR learns a sentence-level semantic representation by distilling knowledge from the cascading model of unsupervised ASR (UASR) and dense text retriever (TDR)
  arXiv  Detail & Related papers  (2024-01-24T14:08:38Z)
• SQUARE: Automatic Question Answering Evaluation using Multiple Positive and Negative References [73.67707138779245]
  We propose a new evaluation metric: SQuArE (Sentence-level QUestion AnsweRing Evaluation) We evaluate SQuArE on both sentence-level extractive (Answer Selection) and generative (GenQA) QA systems.
  arXiv  Detail & Related papers  (2023-09-21T16:51:30Z)
• Reinforced Question Rewriting for Conversational Question Answering [25.555372505026526]
  We develop a model to rewrite conversational questions into self-contained ones. It allows using existing single-turn QA systems to avoid training a CQA model from scratch. We propose using QA feedback to supervise the rewriting model with reinforcement learning.
  arXiv  Detail & Related papers  (2022-10-27T21:23:36Z)
• DUAL: Textless Spoken Question Answering with Speech Discrete Unit Adaptive Learning [66.71308154398176]
  Spoken Question Answering (SQA) has gained research attention and made remarkable progress in recent years. Existing SQA methods rely on Automatic Speech Recognition (ASR) transcripts, which are time and cost-prohibitive to collect. This work proposes an ASR transcript-free SQA framework named Discrete Unit Adaptive Learning (DUAL), which leverages unlabeled data for pre-training and is fine-tuned by the SQA downstream task.
  arXiv  Detail & Related papers  (2022-03-09T17:46:22Z)
• Knowledge Distillation for Improved Accuracy in Spoken Question Answering [63.72278693825945]
  We devise a training strategy to perform knowledge distillation from spoken documents and written counterparts. Our work makes a step towards distilling knowledge from the language model as a supervision signal. Experiments demonstrate that our approach outperforms several state-of-the-art language models on the Spoken-SQuAD dataset.
  arXiv  Detail & Related papers  (2020-10-21T15:18:01Z)
• Contextualized Attention-based Knowledge Transfer for Spoken Conversational Question Answering [63.72278693825945]
  Spoken conversational question answering (SCQA) requires machines to model complex dialogue flow. We propose CADNet, a novel contextualized attention-based distillation approach. We conduct extensive experiments on the Spoken-CoQA dataset and demonstrate that our approach achieves remarkable performance.
  arXiv  Detail & Related papers  (2020-10-21T15:17:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.