Toward Interactive Dictation
- URL: http://arxiv.org/abs/2307.04008v1
- Date: Sat, 8 Jul 2023 16:30:13 GMT
- Title: Toward Interactive Dictation
- Authors: Belinda Z. Li, Jason Eisner, Adam Pauls, Sam Thomson
- Abstract summary: We study the feasibility of allowing users to interrupt their dictation with spoken editing commands in open-ended natural language.
To support this flexibility in real-time, a system must incrementally segment and classify spans of speech as either dictation or command, and interpret the spans that are commands.
Experiments show a natural trade-off between model accuracy and latency: a smaller model achieves 30% end-state accuracy with 1.3 seconds of latency, while a larger model achieves 55% end-state accuracy with 7 seconds of latency.
- Score: 27.67813195022947
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Voice dictation is an increasingly important text input modality. Existing
systems that allow both dictation and editing-by-voice restrict their command
language to flat templates invoked by trigger words. In this work, we study the
feasibility of allowing users to interrupt their dictation with spoken editing
commands in open-ended natural language. We introduce a new task and dataset,
TERTiUS, to experiment with such systems. To support this flexibility in
real-time, a system must incrementally segment and classify spans of speech as
either dictation or command, and interpret the spans that are commands. We
experiment with using large pre-trained language models to predict the edited
text, or alternatively, to predict a small text-editing program. Experiments
show a natural trade-off between model accuracy and latency: a smaller model
achieves 30% end-state accuracy with 1.3 seconds of latency, while a larger
model achieves 55% end-state accuracy with 7 seconds of latency.
Related papers
- Gujarati-English Code-Switching Speech Recognition using ensemble
prediction of spoken language [29.058108207186816]
We propose two methods of introducing language specific parameters and explainability in the multi-head attention mechanism.
Despite being unable to reduce WER significantly, our method shows promise in predicting the correct language from just spoken data.
arXiv Detail & Related papers (2024-03-12T18:21:20Z) - Temporal Validity Change Prediction [20.108317515225504]
Existing benchmarking tasks require models to identify the temporal validity duration of a single statement.
In many cases, additional contextual information, such as sentences in a story or posts on a social media profile, can be collected from the available text stream.
We propose Temporal Validity Change Prediction, a natural language processing task benchmarking the capability of machine learning models to detect contextual statements that induce such change.
arXiv Detail & Related papers (2024-01-01T14:58:53Z) - Take the Hint: Improving Arabic Diacritization with
Partially-Diacritized Text [4.863310073296471]
We propose 2SDiac, a multi-source model that can effectively support optional diacritics in input to inform all predictions.
We also introduce Guided Learning, a training scheme to leverage given diacritics in input with different levels of random masking.
arXiv Detail & Related papers (2023-06-06T10:18:17Z) - Finstreder: Simple and fast Spoken Language Understanding with Finite
State Transducers using modern Speech-to-Text models [69.35569554213679]
In Spoken Language Understanding (SLU) the task is to extract important information from audio commands.
This paper presents a simple method for embedding intents and entities into Finite State Transducers.
arXiv Detail & Related papers (2022-06-29T12:49:53Z) - Dict-TTS: Learning to Pronounce with Prior Dictionary Knowledge for
Text-to-Speech [88.22544315633687]
Polyphone disambiguation aims to capture accurate pronunciation knowledge from natural text sequences for reliable Text-to-speech systems.
We propose Dict-TTS, a semantic-aware generative text-to-speech model with an online website dictionary.
Experimental results in three languages show that our model outperforms several strong baseline models in terms of pronunciation accuracy.
arXiv Detail & Related papers (2022-06-05T10:50:34Z) - Zero-Shot Text-to-Speech for Text-Based Insertion in Audio Narration [62.75234183218897]
We propose a one-stage context-aware framework to generate natural and coherent target speech without any training data of the speaker.
We generate the mel-spectrogram of the edited speech with a transformer-based decoder.
It outperforms a recent zero-shot TTS engine by a large margin.
arXiv Detail & Related papers (2021-09-12T04:17:53Z) - Direct speech-to-speech translation with discrete units [64.19830539866072]
We present a direct speech-to-speech translation (S2ST) model that translates speech from one language to speech in another language without relying on intermediate text generation.
We propose to predict the self-supervised discrete representations learned from an unlabeled speech corpus instead.
When target text transcripts are available, we design a multitask learning framework with joint speech and text training that enables the model to generate dual mode output (speech and text) simultaneously in the same inference pass.
arXiv Detail & Related papers (2021-07-12T17:40:43Z) - Ctrl-P: Temporal Control of Prosodic Variation for Speech Synthesis [68.76620947298595]
Text does not fully specify the spoken form, so text-to-speech models must be able to learn from speech data that vary in ways not explained by the corresponding text.
We propose a model that generates speech explicitly conditioned on the three primary acoustic correlates of prosody.
arXiv Detail & Related papers (2021-06-15T18:03:48Z) - What shall we do with an hour of data? Speech recognition for the un-
and under-served languages of Common Voice [0.20774268785384567]
This report describes the methods and results of a three-week sprint to produce deployable speech recognition models for 31 under-served languages of the Common Voice project.
arXiv Detail & Related papers (2021-05-10T21:16:28Z) - Bridging the Modality Gap for Speech-to-Text Translation [57.47099674461832]
End-to-end speech translation aims to translate speech in one language into text in another language via an end-to-end way.
Most existing methods employ an encoder-decoder structure with a single encoder to learn acoustic representation and semantic information simultaneously.
We propose a Speech-to-Text Adaptation for Speech Translation model which aims to improve the end-to-end model performance by bridging the modality gap between speech and text.
arXiv Detail & Related papers (2020-10-28T12:33:04Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.