See what I'm saying? Comparing Intelligent Personal Assistant use for Native and Non-Native Language Speakers

• URL: http://arxiv.org/abs/2006.06328v1
• Date: Thu, 11 Jun 2020 11:03:49 GMT
• Title: See what I'm saying? Comparing Intelligent Personal Assistant use for Native and Non-Native Language Speakers
• Abstract summary: We aim to understand how IPAs currently support or hinder users. Interviews revealed that L2 speakers prioritised utterance planning around perceived linguistic limitations. L2 speakers see IPAs as insensitive to linguistic needs resulting in failed interaction. We discuss the need to tailor the IPA experience for L2 users, emphasising visual feedback whilst reducing the burden of language production.
• Score: 12.021912017203842
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Limited linguistic coverage for Intelligent Personal Assistants (IPAs) means that many interact in a non-native language. Yet we know little about how IPAs currently support or hinder these users. Through native (L1) and non-native (L2) English speakers interacting with Google Assistant on a smartphone and smart speaker, we aim to understand this more deeply. Interviews revealed that L2 speakers prioritised utterance planning around perceived linguistic limitations, as opposed to L1 speakers prioritising succinctness because of system limitations. L2 speakers see IPAs as insensitive to linguistic needs resulting in failed interaction. L2 speakers clearly preferred using smartphones, as visual feedback supported diagnoses of communication breakdowns whilst allowing time to process query results. Conversely, L1 speakers preferred smart speakers, with audio feedback being seen as sufficient. We discuss the need to tailor the IPA experience for L2 users, emphasising visual feedback whilst reducing the burden of language production.

Related papers

• Building Multilingual Bridges: Data Mixing as the Pillar of Generalization for In-Language Reasoning [75.46846056993418]
  Reasoning language models primarily reason in English regardless of the language they are prompted in.<n>This is inaccessible for non-English-speaking users, risks losing the intent of the original question, and forgoes knowledge more readily expressed in the target language.<n>In this work, we advance L2 reasoning, the ability of a model to reason consistently in the language of the user's prompt, thus building an in-language bridge between the prompt and the answer.
  arXiv  Detail & Related papers  (2026-09-09T16:56:25Z)
• Deaf and Hard of Hearing Access to Intelligent Personal Assistants: Comparison of Voice-Based Options with an LLM-Powered Touch Interface [7.968583771762071]
  We investigate intelligent personal assistants (IPAs) accessibility for deaf and hard of hearing (DHH) people.<n>The inability of IPAs to understand diverse accents including deaf speech renders them largely inaccessible to non-signing and speaking DHH individuals.
  arXiv  Detail & Related papers  (2026-01-21T17:33:00Z)
• ELEGANCE: Efficient LLM Guidance for Audio-Visual Target Speech Extraction [88.41471266579333]
  We propose ELEGANCE, a novel framework that incorporates linguistic knowledge from large language models (LLMs) into AV-TSE models.<n> Comprehensive experiments with RoBERTa, Qwen3-0.6B, and Qwen3-4B on two AV-TSE backbones show significant improvements.
  arXiv  Detail & Related papers  (2025-11-09T08:50:11Z)
• You Sound a Little Tense: L2 Tailored Clear TTS Using Durational Vowel Properties [3.0995661604265385]
  We present the first text-to-speech (TTS) system tailored to second language (L2) speakers.<n>We use duration differences between American English tense (longer) and lax (shorter) vowels to create a "clarity mode" for Matcha-TTS.
  arXiv  Detail & Related papers  (2025-06-29T18:55:05Z)
• A Pilot Study of Applying Sequence-to-Sequence Voice Conversion to Evaluate the Intelligibility of L2 Speech Using a Native Speaker's Shadowings [12.29892010056753]
  An ideal form of feedback for L2 speakers should be so fine-grained that it enables them to detect and diagnose unintelligible parts of utterances. This pilot study utilizes a unique semi-parallel dataset composed of non-native speakers' (L2) reading aloud, shadowing of native speakers (L1) and their script-shadowing utterances. We explore the technical possibility of replicating the process of an L1 speaker's shadowing L2 speech using Voice Conversion techniques, to create a virtual shadower system.
  arXiv  Detail & Related papers  (2024-10-03T06:24:56Z)
• An Initial Investigation of Language Adaptation for TTS Systems under Low-resource Scenarios [76.11409260727459]
  This paper explores the language adaptation capability of ZMM-TTS, a recent SSL-based multilingual TTS system. We demonstrate that the similarity in phonetics between the pre-training and target languages, as well as the language category, affects the target language's adaptation performance.
  arXiv  Detail & Related papers  (2024-06-13T08:16:52Z)
• What Do Dialect Speakers Want? A Survey of Attitudes Towards Language Technology for German Dialects [60.8361859783634]
  We survey speakers of dialects and regional languages related to German. We find that respondents are especially in favour of potential NLP tools that work with dialectal input.
  arXiv  Detail & Related papers  (2024-02-19T09:15:28Z)
• L1-aware Multilingual Mispronunciation Detection Framework [10.15106073866792]
  This paper introduces a novel multilingual MDD architecture, L1-MultiMDD, enriched with L1-aware speech representation. An end-to-end speech encoder is trained on the input signal and its corresponding reference phoneme sequence. Experiments demonstrate the effectiveness of the proposed L1-MultiMDD framework on both seen -- L2-ARTIC, LATIC, and AraVoiceL2v2; and unseen -- EpaDB and Speechocean762 datasets.
  arXiv  Detail & Related papers  (2023-09-14T13:53:17Z)
• BiPhone: Modeling Inter Language Phonetic Influences in Text [12.405907573933378]
  A large number of people are forced to use the Web in a language they have low literacy in due to technology asymmetries. Written text in the second language (L2) from such users often contains a large number of errors that are influenced by their native language (L1) We propose a method to mine phoneme confusions (sounds in L2 that an L1 speaker is likely to conflate) for pairs of L1 and L2. These confusions are then plugged into a generative model (Bi-Phone) for synthetically producing corrupted L2 text.
  arXiv  Detail & Related papers  (2023-07-06T22:31:55Z)
• Still no evidence for an effect of the proportion of non-native speakers on language complexity -- A response to Kauhanen, Einhaus & Walkden (2023) [91.3755431537592]
  Kauhanen, Einhaus & Walkden criticise results presented in one of my papers. I show in this rejoinder that both points of criticism are explicitly mentioned and analysed in my paper.
  arXiv  Detail & Related papers  (2023-04-29T09:58:32Z)
• Bilingual by default: Voice Assistants and the role of code-switching in creating a bilingual user experience [0.5047889990543175]
  We highlight the language production challenges faced in VA interaction for bilingual users. We argue that, by facilitating phenomena seen in bilingual interaction, such as code-switching, we can foster a more inclusive and improved user experience.
  arXiv  Detail & Related papers  (2022-06-20T13:26:48Z)
• Adapt-and-Adjust: Overcoming the Long-Tail Problem of Multilingual Speech Recognition [58.849768879796905]
  We propose Adapt-and-Adjust (A2), a transformer-based multi-task learning framework for end-to-end multilingual speech recognition. The A2 framework overcomes the long-tail problem via three techniques: (1) exploiting a pretrained multilingual language model (mBERT) to improve the performance of low-resource languages; (2) proposing dual adapters consisting of both language-specific and language-agnostic adaptation with minimal additional parameters; and (3) overcoming the class imbalance, either by imposing class priors in the loss during training or adjusting the logits of the softmax output during inference.
  arXiv  Detail & Related papers  (2020-12-03T03:46:16Z)
• Acoustics Based Intent Recognition Using Discovered Phonetic Units for Low Resource Languages [51.0542215642794]
  We propose a novel acoustics based intent recognition system that uses discovered phonetic units for intent classification. We present results for two languages families - Indic languages and Romance languages, for two different intent recognition tasks.
  arXiv  Detail & Related papers  (2020-11-07T00:35:31Z)
• Speaker Independent and Multilingual/Mixlingual Speech-Driven Talking Head Generation Using Phonetic Posteriorgrams [58.617181880383605]
  In this work, we propose a novel approach using phonetic posteriorgrams. Our method doesn't need hand-crafted features and is more robust to noise compared to recent approaches. Our model is the first to support multilingual/mixlingual speech as input with convincing results.
  arXiv  Detail & Related papers  (2020-06-20T16:32:43Z)
• Mental Workload and Language Production in Non-Native Speaker IPA Interaction [12.021912017203842]
  We present a mixed-design experiment, wherein native (L1) and non-native (L2) English speakers completed tasks with IPAs through smartphones and smart speakers. We found significantly higher mental workload for L2 speakers during IPA interactions.
  arXiv  Detail & Related papers  (2020-06-11T11:06:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.