Using Pause Information for More Accurate Entity Recognition
- URL: http://arxiv.org/abs/2109.13222v1
- Date: Mon, 27 Sep 2021 17:47:21 GMT
- Title: Using Pause Information for More Accurate Entity Recognition
- Authors: Sahas Dendukuri, Pooja Chitkara, Joel Ruben Antony Moniz, Xiao Yang,
Manos Tsagkias, Stephen Pulman
- Abstract summary: We show that linguistic observation on pauses can be used to improve accuracy in machine-learnt language understanding tasks.
In contrast to text-based NLU, we apply pause duration to enrich contextual embeddings.
Results show that our proposed novel embeddings improve the relative error rate by up to 8% consistently across three domains for French.
- Score: 6.912121934692421
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Entity tags in human-machine dialog are integral to natural language
understanding (NLU) tasks in conversational assistants. However, current
systems struggle to accurately parse spoken queries with the typical use of
text input alone, and often fail to understand the user intent. Previous work
in linguistics has identified a cross-language tendency for longer speech
pauses surrounding nouns as compared to verbs. We demonstrate that the
linguistic observation on pauses can be used to improve accuracy in
machine-learnt language understanding tasks. Analysis of pauses in French and
English utterances from a commercial voice assistant shows the statistically
significant difference in pause duration around multi-token entity span
boundaries compared to within entity spans. Additionally, in contrast to
text-based NLU, we apply pause duration to enrich contextual embeddings to
improve shallow parsing of entities. Results show that our proposed novel
embeddings improve the relative error rate by up to 8% consistently across
three domains for French, without any added annotation or alignment costs to
the parser.
Related papers
- Gujarati-English Code-Switching Speech Recognition using ensemble
prediction of spoken language [29.058108207186816]
We propose two methods of introducing language specific parameters and explainability in the multi-head attention mechanism.
Despite being unable to reduce WER significantly, our method shows promise in predicting the correct language from just spoken data.
arXiv Detail & Related papers (2024-03-12T18:21:20Z) - Mitigating the Linguistic Gap with Phonemic Representations for Robust
Multilingual Language Understanding [27.318574025851994]
Performance gaps between languages are affected by linguistic gaps between those languages.
We present evidence from three cross-lingual tasks that demonstrate the effectiveness of phonemic representation.
arXiv Detail & Related papers (2024-02-22T04:41:52Z) - Syntactic Language Change in English and German: Metrics, Parsers, and Convergences [56.47832275431858]
The current paper looks at diachronic trends in syntactic language change in both English and German, using corpora of parliamentary debates from the last c. 160 years.
We base our observations on five dependencys, including the widely used Stanford Core as well as 4 newer alternatives.
We show that changes in syntactic measures seem to be more frequent at the tails of sentence length distributions.
arXiv Detail & Related papers (2024-02-18T11:46:16Z) - Pixel Sentence Representation Learning [67.4775296225521]
In this work, we conceptualize the learning of sentence-level textual semantics as a visual representation learning process.
We employ visually-grounded text perturbation methods like typos and word order shuffling, resonating with human cognitive patterns, and enabling perturbation to be perceived as continuous.
Our approach is further bolstered by large-scale unsupervised topical alignment training and natural language inference supervision.
arXiv Detail & Related papers (2024-02-13T02:46:45Z) - Explaining Interactions Between Text Spans [50.70253702800355]
Reasoning over spans of tokens from different parts of the input is essential for natural language understanding.
We introduce SpanEx, a dataset of human span interaction explanations for two NLU tasks: NLI and FC.
We then investigate the decision-making processes of multiple fine-tuned large language models in terms of the employed connections between spans.
arXiv Detail & Related papers (2023-10-20T13:52:37Z) - Bilingual End-to-End ASR with Byte-Level Subwords [4.268218327369146]
We study different representations including character-level, byte-level, byte pair encoding (BPE), and byte-level byte pair encoding (BBPE)
We focus on developing a single end-to-end model to support utterance-based bilingual ASR, where speakers do not alternate between two languages in a single utterance but may change languages across utterances.
We find that BBPE with penalty schemes can improve utterance-based bilingual ASR performance by 2% to 5% relative even with smaller number of outputs and fewer parameters.
arXiv Detail & Related papers (2022-05-01T15:01:01Z) - VECO: Variable and Flexible Cross-lingual Pre-training for Language
Understanding and Generation [77.82373082024934]
We plug a cross-attention module into the Transformer encoder to explicitly build the interdependence between languages.
It can effectively avoid the degeneration of predicting masked words only conditioned on the context in its own language.
The proposed cross-lingual model delivers new state-of-the-art results on various cross-lingual understanding tasks of the XTREME benchmark.
arXiv Detail & Related papers (2020-10-30T03:41:38Z) - Pareto Probing: Trading Off Accuracy for Complexity [87.09294772742737]
We argue for a probe metric that reflects the fundamental trade-off between probe complexity and performance.
Our experiments with dependency parsing reveal a wide gap in syntactic knowledge between contextual and non-contextual representations.
arXiv Detail & Related papers (2020-10-05T17:27:31Z) - Reinforcement learning of minimalist grammars [0.5862282909017474]
State-of-the-art language technology scans the acoustically analyzed speech signal for relevant keywords.
Words are then inserted into semantic slots to interpret the user's intent.
A mental lexicon must be acquired by a cognitive agent during interaction with its users.
arXiv Detail & Related papers (2020-04-30T14:25:58Z) - On the Language Neutrality of Pre-trained Multilingual Representations [70.93503607755055]
We investigate the language-neutrality of multilingual contextual embeddings directly and with respect to lexical semantics.
Our results show that contextual embeddings are more language-neutral and, in general, more informative than aligned static word-type embeddings.
We show how to reach state-of-the-art accuracy on language identification and match the performance of statistical methods for word alignment of parallel sentences.
arXiv Detail & Related papers (2020-04-09T19:50:32Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.