Development and multi-center evaluation of domain-adapted speech recognition for human-AI teaming in real-world gastrointestinal endoscopy

• URL: http://arxiv.org/abs/2604.01705v1
• Date: Thu, 02 Apr 2026 07:13:02 GMT
• Title: Development and multi-center evaluation of domain-adapted speech recognition for human-AI teaming in real-world gastrointestinal endoscopy
• Abstract summary: We present EndoASR, a domain-adapted ASR system for real-time deployment in endoscopic settings.<n>In a retrospective evaluation across six endoscopists, EndoASR substantially improves both transcription accuracy and clinical usability.<n>In a prospective multi-center study, EndoASR demonstrates consistent generalization under heterogeneous real-world conditions.
• Score: 10.190034803243288
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automatic speech recognition (ASR) is a critical interface for human-AI interaction in gastrointestinal endoscopy, yet its reliability in real-world clinical settings is limited by domain-specific terminology and complex acoustic conditions. Here, we present EndoASR, a domain-adapted ASR system designed for real-time deployment in endoscopic workflows. We develop a two-stage adaptation strategy based on synthetic endoscopy reports, targeting domain-specific language modeling and noise robustness. In retrospective evaluation across six endoscopists, EndoASR substantially improves both transcription accuracy and clinical usability, reducing character error rate (CER) from 20.52% to 14.14% and increasing medical term accuracy (Med ACC) from 54.30% to 87.59%. In a prospective multi-center study spanning five independent endoscopy centers, EndoASR demonstrates consistent generalization under heterogeneous real-world conditions. Compared with the baseline Paraformer model, CER is reduced from 16.20% to 14.97%, while Med ACC is improved from 61.63% to 84.16%, confirming its robustness in practical deployment scenarios. Notably, EndoASR achieves a real-time factor (RTF) of 0.005, significantly faster than Whisper-large-v3 (RTF 0.055), while maintaining a compact model size of 220M parameters, enabling efficient edge deployment. Furthermore, integration with large language models demonstrates that improved ASR quality directly enhances downstream structured information extraction and clinician-AI interaction. These results demonstrate that domain-adapted ASR can serve as a reliable interface for human-AI teaming in gastrointestinal endoscopy, with consistent performance validated across multi-center real-world clinical settings.

Related papers

• Recovering Clinical Utility Under Differential Privacy: Empirical Validation of Adaptive Federated Aggregation on Heterogeneous Cardiovascular Datasets [0.06372261626436676]
  The present work addresses the gap by validating the FedCVR framework on five publicly available real cardiovascular datasets.<n>Results demonstrate that FedCVR preserves its adaptive advantage on real data, achieving an F1-Score of 79.2% and AUC of 0.96.<n>The measured privacy cost on real data confirms the graceful degradation pattern observed in the synthetic experiments.
  arXiv  Detail & Related papers  (2026-07-07T18:51:13Z)
• Cerebra: A Multidisciplinary AI Board for Multimodal Dementia Characterization and Risk Assessment [56.62016795093786]
  Cerebra is an interactive multi-agent AI team that coordinates specialized agents for EHR, clinical notes, and medical imaging analysis.<n>It supports privacy-preserving deployment by operating on structured representations and remains robust when modalities are incomplete.<n>Cerebra significantly improved expert performance, increasing accuracy by 17.5 percentage points in prospective dementia risk estimation.
  arXiv  Detail & Related papers  (2026-03-23T05:46:45Z)
• A DeepSeek-Powered AI System for Automated Chest Radiograph Interpretation in Clinical Practice [83.11942224668127]
  Janus-Pro-CXR (1B) is a chest X-ray interpretation system based on DeepSeek Janus-Pro model.<n>Our system outperforms state-of-the-art X-ray report generation models in automated report generation.
  arXiv  Detail & Related papers  (2025-12-23T13:26:13Z)
• Cancer-Net PCa-MultiSeg: Multimodal Enhancement of Prostate Cancer Lesion Segmentation Using Synthetic Correlated Diffusion Imaging [55.62977326180104]
  Current deep learning approaches for prostate cancer lesion segmentation achieve limited performance.<n>We investigate synthetic correlated diffusion imaging (CDI$s$) as an enhancement to standard diffusion-based protocols.<n>Our results establish validated integration pathways for CDI$s$ as a practical drop-in enhancement for PCa lesion segmentation tasks.
  arXiv  Detail & Related papers  (2025-11-11T04:16:12Z)
• Longitudinal Vestibular Schwannoma Dataset with Consensus-based Human-in-the-loop Annotations [3.1898695141875772]
  This dataset includes 190 patients, with tumour annotations available for 534 longitudinal contrast-enhanced T1-weighted (T1CE) scans from 184 patients, and non-annotated T2-weighted scans from 6 patients.<n>We show that our approach enables effective and resource-efficient generalisation of automated segmentation models to a target data distribution.<n>It is estimated to enhance efficiency by approximately 37.4% compared to the conventional manual annotation process.
  arXiv  Detail & Related papers  (2025-11-01T09:53:28Z)
• LGE-Guided Cross-Modality Contrastive Learning for Gadolinium-Free Cardiomyopathy Screening in Cine CMR [51.11296719862485]
  We propose a Contrastive Learning and Cross-Modal alignment framework for gadolinium-free cardiomyopathy screening using cine CMR sequences.<n>By aligning the latent spaces of cine CMR and Late Gadolinium Enhancement (LGE) sequences, our model encodes fibrosis-specific pathology into cine CMR embeddings.
  arXiv  Detail & Related papers  (2025-08-23T07:21:23Z)
• A Novel Attention-Augmented Wavelet YOLO System for Real-time Brain Vessel Segmentation on Transcranial Color-coded Doppler [49.03919553747297]
  We propose an AI-powered, real-time CoW auto-segmentation system capable of efficiently capturing cerebral arteries.<n>No prior studies have explored AI-driven cerebrovascular segmentation using Transcranial Color-coded Doppler (TCCD)<n>The proposed AAW-YOLO demonstrated strong performance in segmenting both ipsilateral and contralateral CoW vessels.
  arXiv  Detail & Related papers  (2025-08-19T14:41:22Z)
• Agentic large language models improve retrieval-based radiology question answering [4.208637377704778]
  radiology Retrieval and Reasoning (RaR) is a multi-step retrieval and reasoning framework for radiology question answering.<n>RaR significantly improved mean diagnostic accuracy over zero-shot prompting and conventional online RAG.<n>RaR retrieval reduced hallucinations (mean 9.4%) and retrieved clinically relevant context in 46% of cases.
  arXiv  Detail & Related papers  (2025-08-01T16:18:52Z)
• How Well Can Modern LLMs Act as Agent Cores in Radiology Environments? [54.36730060680139]
  RadA-BenchPlat is an evaluation platform that benchmarks the performance of large language models (LLMs) in radiology environments.<n>The platform also defines ten categories of tools for agent-driven task solving and evaluates seven leading LLMs.
  arXiv  Detail & Related papers  (2024-12-12T18:20:16Z)
• High-precision medical speech recognition through synthetic data and semantic correction: UNITED-MEDASR [1.3810901729134184]
  We introduce United-MedASR, a novel architecture that addresses challenges by integrating synthetic data generation, precision ASR fine-tuning, and semantic enhancement techniques.<n>United-MedASR constructs a specialised medical vocabulary by synthesising data from authoritative sources such as ICD-10, MIMS, and FDA databases.<n>To enhance processing speed, we incorporate Faster Whisper, ensuring streamlined and high-speed ASR performance.
  arXiv  Detail & Related papers  (2024-11-24T17:02:48Z)
• Interactive 3D Segmentation for Primary Gross Tumor Volume in Oropharyngeal Cancer [1.9997842016096374]
  We implement state-of-the-art algorithms and propose a novel two-stage Interactive Click Refinement framework. The 2S-ICR framework achieves a Dice similarity coefficient of 0.713 $pm$ 0.152 without user interaction and 0.824 $pm$ 0.099 after five interactions, outperforming existing methods in both cases.
  arXiv  Detail & Related papers  (2024-09-10T15:58:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.