Not All Fallbacks Are Failures: Understanding and Recovering from Fallbacks in Mobile Voice Assistants

• URL: http://arxiv.org/abs/2608.30738v1
• Date: Mon, 31 Aug 2026 13:06:42 GMT
• Title: Not All Fallbacks Are Failures: Understanding and Recovering from Fallbacks in Mobile Voice Assistants
• Abstract summary: We study fallback handling in a deployed smartwatch-based voice assistant for general health support in everyday environments.<n>Our analysis is based on six months of real-world usage data from more than 500 users, yielding a dataset of 3,030 anonymized, naturally occurring fallback-triggering utterances.
• Score: 50.58363289248172
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Robust understanding of user input is a core requirement for voice assistants deployed in real-world environments. In practice, these systems encounter heterogeneous fallback situations caused by noisy audio input, transcription errors, ambiguous requests, incomplete utterances, or unintended activations. Existing systems typically respond with generic fallback messages, which do not resolve the underlying interaction failure and can degrade user experience. We study fallback handling in a deployed smartwatch-based voice assistant for general health support in everyday environments. Our analysis is based on six months of real-world usage data from more than 500 users, yielding a dataset of 3,030 anonymized, naturally occurring fallback-triggering utterances. We contribute (1) an operational taxonomy and the annotated VoxFallbacks dataset of these interactions, (2) a comparative evaluation of different models within a classification pipeline under practical deployment constraints, and (3) practical lessons for designing robust and cost-efficient fallback mechanisms. Results show that lightweight embedding-based classifiers outperform larger generative models on most classification tasks while requiring substantially fewer computational resources.

Related papers

• Not Every Subject Should Stay: Machine Unlearning for Noisy Engagement Recognition [53.005382593686356]
  Engagement recognition datasets are typically subject-indexed and often contain noisy, subjective supervision.<n>We study this setting through subject-level machine unlearning as a post-hoc sanitization mechanism for engagement recognition.
  arXiv  Detail & Related papers  (2026-05-06T10:03:06Z)
• SpeechParaling-Bench: A Comprehensive Benchmark for Paralinguistic-Aware Speech Generation [46.48800527703154]
  SpeechParaling-Bench is a comprehensive benchmark for paralinguistic-aware speech generation.<n>It expands existing coverage from fewer than 50 to over 100 fine-grained features.<n>It is organized into three progressively challenging tasks: fine-grained control, intra-utterance variation, and context-aware adaptation.
  arXiv  Detail & Related papers  (2026-04-22T17:59:58Z)
• Dynamic analysis enhances issue resolution [53.50448142467294]
  DAIRA (Dynamic Analysis-enhanced Issue Resolution Agent) is an automated repair framework that embeds dynamic analysis into the agent's reasoning cycle.<n> Driven by a Test Tracing-Driven methodology, DAIRA utilizes lightweight monitors to extract critical runtime data.<n>When powered by Gemini 3 Flash Preview, DAIRA establishes a new state-of-the-art (SOTA) performance, achieving a 79.4% resolution rate on the SWE-bench Verified dataset.
  arXiv  Detail & Related papers  (2026-03-23T14:48:54Z)
• Multi-agent Auditory Scene Analysis [0.0]
  Auditory scene analysis (ASA) aims to retrieve information from the acoustic environment, by carrying out three main tasks: sound source location, separation, and classification.<n>Running these tasks linearly increases the overall response time, while making the last tasks highly sensitive to errors of the first task (location)<n>A multi-agent approach is proposed to carry out ASA where the tasks are run in parallel, with feedback loops between them to compensate for local errors.
  arXiv  Detail & Related papers  (2025-07-03T16:16:46Z)
• Towards Robust Few-shot Class Incremental Learning in Audio Classification using Contrastive Representation [1.3586572110652484]
  Few-shot class-incremental learning addresses challenges arising from limited incoming data. We propose supervised contrastive learning to refine the representation space, enhancing discriminative power and leading to better generalization.
  arXiv  Detail & Related papers  (2024-07-27T14:16:25Z)
• The Art of Saying No: Contextual Noncompliance in Language Models [123.383993700586]
  We introduce a comprehensive taxonomy of contextual noncompliance describing when and how models should not comply with user requests. Our taxonomy spans a wide range of categories including incomplete, unsupported, indeterminate, and humanizing requests. To test noncompliance capabilities of language models, we use this taxonomy to develop a new evaluation suite of 1000 noncompliance prompts.
  arXiv  Detail & Related papers  (2024-07-02T07:12:51Z)
• Boosting keyword spotting through on-device learnable user speech characteristics [16.632185274553485]
  Keywords spotting systems for always-on TinyML-constrained applications require on-site tuning. Adapting to the speech peculiarities of target users requires many in-domain samples, often unavailable in real-world scenarios. We propose a novel on-device learning architecture, composed of a pretrained backbone and a user-aware embedding learning the user's speech characteristics.
  arXiv  Detail & Related papers  (2024-03-12T16:41:31Z)
• RLVF: Learning from Verbal Feedback without Overgeneralization [94.19501420241188]
  We study the problem of incorporating verbal feedback without such overgeneralization. We develop a new method Contextualized Critiques with Constrained Preference Optimization (C3PO) Our approach effectively applies verbal feedback to relevant scenarios while preserving existing behaviors for other contexts.
  arXiv  Detail & Related papers  (2024-02-16T18:50:24Z)
• Stateful Offline Contextual Policy Evaluation and Learning [88.9134799076718]
  We study off-policy evaluation and learning from sequential data. We formalize the relevant causal structure of problems such as dynamic personalized pricing. We show improved out-of-sample policy performance in this class of relevant problems.
  arXiv  Detail & Related papers  (2021-10-19T16:15:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.