Bilingual by default: Voice Assistants and the role of code-switching in creating a bilingual user experience

• URL: http://arxiv.org/abs/2206.09765v1
• Date: Mon, 20 Jun 2022 13:26:48 GMT
• Title: Bilingual by default: Voice Assistants and the role of code-switching in creating a bilingual user experience
• Abstract summary: We highlight the language production challenges faced in VA interaction for bilingual users. We argue that, by facilitating phenomena seen in bilingual interaction, such as code-switching, we can foster a more inclusive and improved user experience.
• Score: 0.5047889990543175
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Conversational User Interfaces such as Voice Assistants are hugely popular. Yet they are designed to be monolingual by default, lacking support for, or sensitivity to, the bilingual dialogue experience. In this provocation paper, we highlight the language production challenges faced in VA interaction for bilingual users. We argue that, by facilitating phenomena seen in bilingual interaction, such as code-switching, we can foster a more inclusive and improved user experience for bilingual users. We also explore ways that this might be achieved, through the support of multiple language recognition as well as being sensitive to the preferences of code-switching in speech output.

Related papers

• Strategies of Code-switching in Human-Machine Dialogs [3.417612970260323]
  Most people are multilingual, and most multilinguals code-switch.<n>We developed a bot capable of completing a Map Task with human participants using code-switched Spanish and English.
  arXiv  Detail & Related papers  (2025-08-10T12:41:46Z)
• Code-switching in text and speech reveals information-theoretic audience design [5.3329709073809095]
  We use language modeling to investigate the factors that influence code-switching. Code-switching occurs when a speaker alternates between one language variety (the primary language) and another (the secondary language)
  arXiv  Detail & Related papers  (2024-08-08T17:14:12Z)
• Zero Resource Code-switched Speech Benchmark Using Speech Utterance Pairs For Multiple Spoken Languages [49.6922490267701]
  We introduce a new zero resource code-switched speech benchmark designed to assess the code-switching capabilities of self-supervised speech encoders. We showcase a baseline system of language modeling on discrete units to demonstrate how the code-switching abilities of speech encoders can be assessed.
  arXiv  Detail & Related papers  (2023-10-04T17:58:11Z)
• Enhancing Multilingual Speech Recognition through Language Prompt Tuning and Frame-Level Language Adapter [15.039113587886225]
  We propose two simple and parameter-efficient methods to enhance language-configurable and language-agnostic multilingual speech recognition. Our experiments demonstrate significant performance improvements across seven languages using our proposed methods.
  arXiv  Detail & Related papers  (2023-09-18T02:51:59Z)
• Hindi as a Second Language: Improving Visually Grounded Speech with Semantically Similar Samples [89.16814518860357]
  The objective of this work is to explore the learning of visually grounded speech models (VGS) from multilingual perspective. Our key contribution in this work is to leverage the power of a high-resource language in a bilingual visually grounded speech model to improve the performance of a low-resource language.
  arXiv  Detail & Related papers  (2023-03-30T16:34:10Z)
• Code-Switching without Switching: Language Agnostic End-to-End Speech Translation [68.8204255655161]
  We treat speech recognition and translation as one unified end-to-end speech translation problem. By training LAST with both input languages, we decode speech into one target language, regardless of the input language.
  arXiv  Detail & Related papers  (2022-10-04T10:34:25Z)
• LAE: Language-Aware Encoder for Monolingual and Multilingual ASR [87.74794847245536]
  A novel language-aware encoder (LAE) architecture is proposed to handle both situations by disentangling language-specific information. Experiments conducted on Mandarin-English code-switched speech suggest that the proposed LAE is capable of discriminating different languages in frame-level.
  arXiv  Detail & Related papers  (2022-06-05T04:03:12Z)
• Code Switched and Code Mixed Speech Recognition for Indic languages [0.0]
  Training multilingual automatic speech recognition (ASR) systems is challenging because acoustic and lexical information is typically language specific. We compare the performance of end to end multilingual speech recognition system to the performance of monolingual models conditioned on language identification (LID) We also propose a similar technique to solve the Code Switched problem and achieve a WER of 21.77 and 28.27 over Hindi-English and Bengali-English respectively.
  arXiv  Detail & Related papers  (2022-03-30T18:09:28Z)
• Joint Modeling of Code-Switched and Monolingual ASR via Conditional Factorization [75.98664099579392]
  We propose a general framework to jointly model the likelihoods of the monolingual and code-switch sub-tasks that comprise bilingual speech recognition. We demonstrate the efficacy of our proposed model on bilingual Mandarin-English speech recognition across both monolingual and code-switched corpora.
  arXiv  Detail & Related papers  (2021-11-29T23:14:54Z)
• Transformer-Transducers for Code-Switched Speech Recognition [23.281314397784346]
  We present an end-to-end ASR system using a transformer-transducer model architecture for code-switched speech recognition. First, we introduce two auxiliary loss functions to handle the low-resource scenario of code-switching. Second, we propose a novel mask-based training strategy with language ID information to improve the label encoder training towards intra-sentential code-switching.
  arXiv  Detail & Related papers  (2020-11-30T17:27:41Z)
• VECO: Variable and Flexible Cross-lingual Pre-training for Language Understanding and Generation [77.82373082024934]
  We plug a cross-attention module into the Transformer encoder to explicitly build the interdependence between languages. It can effectively avoid the degeneration of predicting masked words only conditioned on the context in its own language. The proposed cross-lingual model delivers new state-of-the-art results on various cross-lingual understanding tasks of the XTREME benchmark.
  arXiv  Detail & Related papers  (2020-10-30T03:41:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.