Bridging the Usability Gap: Lessons from Interpreting Studies for Machine Interpreting Design

• URL: http://arxiv.org/abs/2606.16009v2
• Date: Tue, 16 Jun 2026 12:53:59 GMT
• Title: Bridging the Usability Gap: Lessons from Interpreting Studies for Machine Interpreting Design
• Abstract summary: Machine interpreting (MI) is the live, real-time application of speech translation.<n>Yet the user experience remains far inferior to interpreter-mediated communication.<n>This paper defines MI as a distinct subfield of speech translation.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine interpreting (MI), the live, real-time application of speech translation, has achieved remarkable progress on standard benchmarks, with some systems approaching human parity on textual fidelity. Yet the user experience remains far inferior to interpreter-mediated communication, revealing what we term the accuracy illusion: systems that appear accurate on paper but fail in practice to support smooth, goal-oriented interaction. This paper defines MI as a distinct subfield of speech translation, with its own characteristics and the need for evaluation methods grounded in communicative effectiveness rather than isolated fidelity metrics. Drawing on insights from interpreting studies, we identify critical dimensions of professional interpreting practice that are overlooked by current systems, and consolidate them into three interdependent design priorities for future MI: agency (context-sensitive initiative and repair), grounding (multimodal and discourse-level situational awareness), and experience (adaptive improvement through real interaction). Together, these priorities chart a path toward closing the usability gap and enabling systems that can sustain authentic multilingual communication in real time.

Related papers

• When the Target Domain Changes: AI-Mediated Construct Drift in High-Stakes English Language AssessmenW [1.2333280983346018]
  High-stakes English proficiency tests treat standardized, unaided performance as evidence for score interpretations about academic English proficiency.<n>This conceptual validity argument reframes AI as a score-interpretation problem in high-stakes language testing.<n>The paper argues that score interpretations should be narrowed and supplemented when used to support claims about AI-mediated academic communication.
  arXiv  Detail & Related papers  (2026-07-13T08:07:30Z)
• Ouvia: A User-centered Framework for Measuring Usability of Speech Translation in Real-World Communication Scenarios [63.167237308464536]
  Speech translation (ST) is increasingly adopted in user applications, yet its evaluation focuses on decontextualized testbeds and holistic quality.<n>We introduce Ouvia, an evaluation framework for measuring user-perceived usability of speech translation outputs in real-world settings.
  arXiv  Detail & Related papers  (2026-06-04T13:52:21Z)
• Interactive ASR: Towards Human-Like Interaction and Semantic Coherence Evaluation for Agentic Speech Recognition [52.63676763985825]
  We propose a semantic-aware evaluation metric to assess recognition quality beyond token-level accuracy.<n>We design an agent framework to simulate human-like multi-turn interaction, enabling iterative refinement of recognition outputs.<n>We will release the code to facilitate future research in interactive and agentic ASR.
  arXiv  Detail & Related papers  (2026-04-10T09:02:42Z)
• When Contextual Inference Fails: Cancelability in Interactive Instruction Following [51.2195840589474]
  We introduce Build What I Mean, an interactive benchmark for contextual meaning construction.<n>In BWIM, models must resolve ambiguity by either performing a contextual inference or requesting clarification at a small communication cost.<n>We observe suboptimal strategies, such as partner-blind over-clarification and question-averse guessing under uncertainty.
  arXiv  Detail & Related papers  (2026-03-20T14:46:59Z)
• SignVLA: A Gloss-Free Vision-Language-Action Framework for Real-Time Sign Language-Guided Robotic Manipulation [1.4175612723267692]
  We present the first sign language-driven Vision-Language-Action (VLA) framework for intuitive human-robot interaction.<n>Unlike conventional approaches that rely on gloss annotations as intermediate supervision, the proposed system adopts a gloss-free paradigm.<n>We focus on a real-time alphabet-level finger-spelling interface that provides a robust and low-latency communication channel for robotic control.
  arXiv  Detail & Related papers  (2026-02-26T01:16:27Z)
• Understanding the Modality Gap: An Empirical Study on the Speech-Text Alignment Mechanism of Large Speech Language Models [12.263637152835713]
  End-to-end Large Speech Language Models (LSLMs) have demonstrated impressive conversational generation abilities.<n>We analyze both coarse- and fine-grained text and speech representations.<n>We find that representation similarity is strongly correlated with the modality gap.
  arXiv  Detail & Related papers  (2025-10-14T03:34:38Z)
• Vision-Grounded Machine Interpreting: Improving the Translation Process through Visual Cues [0.0]
  Vision-Grounded Interpreting (VGI) is a novel approach designed to address the limitations of unimodal machine interpreting.<n>We present a prototype system that integrates a vision-language model to process both speech and visual input from a webcam.<n>To evaluate the effectiveness of this approach, we constructed a hand-crafted diagnostic corpus targeting three types of ambiguity.
  arXiv  Detail & Related papers  (2025-09-28T16:25:33Z)
• Towards Inclusive Communication: A Unified Framework for Generating Spoken Language from Sign, Lip, and Audio [52.859261069569165]
  We propose the first unified framework capable of handling diverse combinations of sign language, lip movements, and audio for spoken-language text generation.<n>We focus on three main objectives: (i) designing a unified, modality-agnostic architecture capable of effectively processing heterogeneous inputs; (ii) exploring the underexamined synergy among modalities, particularly the role of lip movements as non-manual cues in sign language comprehension; and (iii) achieving performance on par with or better than state-of-the-art models specialized for individual tasks.
  arXiv  Detail & Related papers  (2025-08-28T06:51:42Z)
• Language of Persuasion and Misrepresentation in Business Communication: A Textual Detection Approach [0.0]
  Business communication digitisation has reorganised the process of persuasive discourse.<n>This inquiry synthesises classical rhetoric and communication psychology with linguistic theory and empirical studies.
  arXiv  Detail & Related papers  (2025-08-13T16:38:31Z)
• How "Real" is Your Real-Time Simultaneous Speech-to-Text Translation System? [7.252894835396412]
  Simultaneous speech-to-text translation (SimulST) translates source-language speech into target-language text concurrently with the speaker's speech, ensuring low latency for better user comprehension.<n>Despite its intended application to unbounded speech, most research has focused on human pre-segmented speech, simplifying the task and overlooking significant challenges.
  arXiv  Detail & Related papers  (2024-12-24T15:26:31Z)
• Cognitive Semantic Communication Systems Driven by Knowledge Graph: Principle, Implementation, and Performance Evaluation [74.38561925376996]
  Two cognitive semantic communication frameworks are proposed for the single-user and multiple-user communication scenarios. An effective semantic correction algorithm is proposed by mining the inference rule from the knowledge graph. For the multi-user cognitive semantic communication system, a message recovery algorithm is proposed to distinguish messages of different users.
  arXiv  Detail & Related papers  (2023-03-15T12:01:43Z)
• Common Language for Goal-Oriented Semantic Communications: A Curriculum Learning Framework [66.81698651016444]
  A comprehensive semantic communications framework is proposed for enabling goal-oriented task execution. A novel top-down framework that combines curriculum learning (CL) and reinforcement learning (RL) is proposed to solve this problem. Simulation results show that the proposed CL method outperforms traditional RL in terms of convergence time, task execution time, and transmission cost during training.
  arXiv  Detail & Related papers  (2021-11-15T19:13:55Z)
• FILIP: Fine-grained Interactive Language-Image Pre-Training [106.19474076935363]
  Fine-grained Interactive Language-Image Pre-training achieves finer-level alignment through a cross-modal late interaction mechanism. We construct a new large-scale image-text pair dataset called FILIP300M for pre-training. Experiments show that FILIP achieves state-of-the-art performance on multiple downstream vision-language tasks.
  arXiv  Detail & Related papers  (2021-11-09T17:15:38Z)
• Towards the evaluation of simultaneous speech translation from a communicative perspective [0.0]
  We present the results of an experiment aimed at evaluating the quality of a simultaneous speech translation engine. We found better performance for the human interpreters in terms of intelligibility, while the machine performs slightly better in terms of informativeness.
  arXiv  Detail & Related papers  (2021-03-15T13:09:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.