Ontology Memory-Augmented ASR Correction for Long Text-Speech Interleaved Conversations

• URL: http://arxiv.org/abs/2606.13464v2
• Date: Fri, 12 Jun 2026 04:13:39 GMT
• Title: Ontology Memory-Augmented ASR Correction for Long Text-Speech Interleaved Conversations
• Abstract summary: We propose a memory-augmented ASR correction framework for long text-speech interleaved conversations.<n>We show that our method improves over direct correction in 9 out of 10 paired backbone-setting combinations.
• Score: 49.31192454394301
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automatic speech recognition (ASR) correction has traditionally focused on isolated utterances or short local contexts. However, as text and speech become increasingly interleaved in long interactions, ASR correction requires conversation-level contextual evidence. Existing ASR correction methods often rely on the current hypothesis or concatenate raw dialogue history. In such contexts, sparse correction evidence can be difficult to locate amid redundancy and noise. Addressing these challenges, we propose an ontology memory-augmented ASR correction framework for long text-speech interleaved conversations. The framework organizes preceding interaction history into a dynamically updatable ontology memory, where entities, terminology, surface variants, potential ASR confusions, and semantic relations are stored as retrievable nodes for context-grounded correction. To evaluate this setting, we construct RAMC-Corr, a dataset derived from MAGIC-RAMC for long-range ASR correction with grounded context. Experiments on RAMC-Corr show that our method improves over direct correction in 9 out of 10 paired backbone-setting combinations and encourages more selective and evidence-grounded corrections for context-dependent ASR errors.

Related papers

• DiaRelay: Relaying Dialogue Context with a Constant-Size Memory for Emotion Recognition in Conversation [27.460834617156028]
  Emotion Recognition in Conversation (ERC) requires models to identify subtle emotional cues that are often distributed across distant dialogue turns.<n>We propose a lightweight adapter, DiaRelay, to enable LLMs to explicitly maintain a dialogue-level memory for accurate ERC.<n>We show that DiaRelay can achieve SOTA weighted F1 and accuracy on MELD while obtaining competitive results on IEMOCAP with only an extra 7.1M trainable parameters.
  arXiv  Detail & Related papers  (2026-08-24T03:09:15Z)
• Rethinking Speech-LLM Integration for ASR: Effective Joint Speech-Text Training by Interleaving [54.66679099886108]
  Speech-LLM integration has shown promising results by leveraging extensive textual pretraining, yet its specific benefits for automatic speech recognition (ASR) remain unclear.<n>We propose Joint Speech-Text Interleaved Pretraining (JSTIP), an ASR-oriented pretraining strategy that constructs word-level and segment-level interleaved speech-text sequences.<n> Experiments on 38k hours of ASR data show consistent entity accuracy improvement compared to ASR-only and joint speech-text training baselines.
  arXiv  Detail & Related papers  (2026-07-02T05:42:01Z)
• Towards Human-Like Interactive Speech Recognition With Agentic Correction and Semantic Evaluation [53.844308305341166]
  We propose textbfAgentic ASR, a closed-loop framework that combines a single-pass ASR front-end with semantic correction, intent routing, and reasoning-based editing.<n> Experiments on multilingual, named-entity-intensive, and code-switching benchmarks show that iterative interaction consistently reduces semantic errors.
  arXiv  Detail & Related papers  (2026-05-28T06:23:31Z)
• Training-Free Intelligibility-Guided Observation Addition for Noisy ASR [57.74127683005929]
  This paper proposes an intelligibility-guided observation addition (OA) method to improve speech recognition in noisy environments.<n>Experiments across diverse SE-ASR combinations and datasets demonstrate strong robustness and improvements over existing OA baselines.
  arXiv  Detail & Related papers  (2026-02-24T14:46:54Z)
• Semantic Anchoring in Agentic Memory: Leveraging Linguistic Structures for Persistent Conversational Context [0.0]
  We propose a hybrid agentic memory architecture that enriches vector-based storage with explicit linguistic cues to improve recall of nuanced, context-rich exchanges.<n>Experiments on adapted long-term dialogue datasets show that semantic anchoring improves factual recall and discourse coherence by up to 18% over strong RAG baselines.
  arXiv  Detail & Related papers  (2025-08-18T05:14:48Z)
• Contextual ASR Error Handling with LLMs Augmentation for Goal-Oriented Conversational AI [27.56203179880491]
  General-purpose automatic speech recognition (ASR) systems do not always perform well in goal-oriented dialogue.<n>We extend correction to tasks that have no prior user data and exhibit linguistic flexibility such as lexical and syntactic variations.
  arXiv  Detail & Related papers  (2025-01-10T17:35:06Z)
• Spelling Correction through Rewriting of Non-Autoregressive ASR Lattices [8.77712061194924]
  We present a finite-state transducer (FST) technique for rewriting wordpiece lattices generated by Transformer-based CTC models. Our algorithm performs grapheme-to-phoneme (G2P) conversion directly from wordpieces into phonemes, avoiding explicit word representations. We achieved up to a 15.2% relative reduction in sentence error rate (SER) on a test set with contextually relevant entities.
  arXiv  Detail & Related papers  (2024-09-24T21:42:25Z)
• Enhancing Dialogue Speech Recognition with Robust Contextual Awareness via Noise Representation Learning [6.363223418619587]
  We introduce Context Noise Representation Learning (CNRL) to enhance robustness against noisy context, ultimately improving dialogue speech recognition accuracy. Based on the evaluation of speech dialogues, our method shows superior results compared to baselines.
  arXiv  Detail & Related papers  (2024-08-12T10:21:09Z)
• Towards interfacing large language models with ASR systems using confidence measures and prompting [54.39667883394458]
  This work investigates post-hoc correction of ASR transcripts with large language models (LLMs) To avoid introducing errors into likely accurate transcripts, we propose a range of confidence-based filtering methods. Our results indicate that this can improve the performance of less competitive ASR systems.
  arXiv  Detail & Related papers  (2024-07-31T08:00:41Z)
• Crossmodal ASR Error Correction with Discrete Speech Units [16.58209270191005]
  We propose a post-ASR processing approach for ASR Error Correction (AEC) We explore pre-training and fine-tuning strategies and uncover an ASR domain discrepancy phenomenon. We propose the incorporation of discrete speech units to align with and enhance the word embeddings for improving AEC quality.
  arXiv  Detail & Related papers  (2024-05-26T19:58:38Z)
• A Comparative Study on Speaker-attributed Automatic Speech Recognition in Multi-party Meetings [53.120885867427305]
  Three approaches are evaluated for speaker-attributed automatic speech recognition (SA-ASR) in a meeting scenario. The WD-SOT approach achieves 10.7% relative reduction on averaged speaker-dependent character error rate (SD-CER) The TS-ASR approach also outperforms the FD-SOT approach and brings 16.5% relative average SD-CER reduction.
  arXiv  Detail & Related papers  (2022-03-31T06:39:14Z)
• Contextualized Attention-based Knowledge Transfer for Spoken Conversational Question Answering [63.72278693825945]
  Spoken conversational question answering (SCQA) requires machines to model complex dialogue flow. We propose CADNet, a novel contextualized attention-based distillation approach. We conduct extensive experiments on the Spoken-CoQA dataset and demonstrate that our approach achieves remarkable performance.
  arXiv  Detail & Related papers  (2020-10-21T15:17:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.