Voice Memory for Agentic Speech Recognition

• URL: http://arxiv.org/abs/2607.26410v1
• Date: Wed, 29 Jul 2026 02:42:56 GMT
• Title: Voice Memory for Agentic Speech Recognition
• Abstract summary: We present Voice Memory, a inference-only scheme for agentic speech recognition.<n>Asynchronously, a score-gated example revises that file through bounded edits.<n>Across ten HyPoradise domains with an open corrector, Voice Memory, weighted word error rate from 8.36% to 7.52%.
• Score: 66.69623133635868
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present Voice Memory, a inference-only scheme for agentic speech recognition: at stream time, a frozen corrector reads a single per-domain memory.md and decides per utterance whether to act on the hypothesis or abstain and keep the 1-best. Asynchronously, a score-gated optimizer revises that file through bounded edits, accepting an edit only when it strictly improves a held-out score. Extended from classical ASR-LM framework, we refer this split the listener-thinker architecture; the two roles are coupled only through the memory, so no weights change and the learned skill stays auditable and portable. Restraint turns out to be the operative skill this loop discovers: unconstrained generative error correction (GER) over-corrects, breaking correct tokens on up to 64% of its edits on financial news, and Voice Memory, reduces this rate to 35%. Across ten HyPoradise domains with an open corrector, Voice Memory, lowers weighted word error rate from 8.36% to 7.52% (7.47% with three added in-context examples) without regressing any dataset below its 1-best baseline; gains concentrate where recoverable headroom is largest, including air-travel commands (8.40% to 3.40%) and noisy far-field speech (CHiME-4, 12.69% to 10.46%). The memory transfers across corrector families and adds zero parameters to the inference path. A demo and example code are provided for future studies.

Related papers

• X-AuT: Progressive Audio-Encoder Compression for Speech LLMs with Cross-Scale Distillation [34.99298958122614]
  X-AuT is a progressive framework that selects layer combinations through short behavioral probes.<n>It restores the pruned model through representation alignment, cross-scale distillation, scheduled student-policy supervision, and LoRA finetuning.<n>On ten public Chinese--English benchmarks, compressing Qwen3-ASR-0.6B from 18 to 16 audio-encoder layers reduces macro-average error from 5.61% to 5.27%.
  arXiv  Detail & Related papers  (2026-09-10T11:43:17Z)
• From Scores to Evidence: Auditable Decisions Can Improve Speech Deepfake Detection [12.159476982130286]
  Speech deepfakes can mimic a speaker's voice convincingly enough to deceive listeners and automated systems.<n>Most detectors end with one score per utterance, but it says little about why a borderline item should be trusted, deferred, or reviewed.<n>We answer this question with an auditable decision record that carries four aligned cues into a late calibration step.
  arXiv  Detail & Related papers  (2026-09-08T15:38:57Z)
• Ground Truth First: A Longitudinal Evaluation Instrument for Agent Memory, and the Tenure Crossover in Memory-Architecture Rankings [0.0]
  A seeded life-script sampler emits facts with validity intervals volatility classes, and source channels before any text exists.<n>A verifier confirms every planted fact and questions are instantiated mechanically from the script.<n>The synthetic, fictionalized corpus embeds features absent from the benchmarks we survey.
  arXiv  Detail & Related papers  (2026-07-24T04:19:45Z)
• Reclaim Evaluation: A Lossy Memory Is Worse Than an Empty One [0.0]
  A language model's memory can be worse than no memory at all.<n>A memory that keeps a wrong conclusion but drops the work behind it makes the model emit the stale value as a confident answer.
  arXiv  Detail & Related papers  (2026-06-24T06:24:53Z)
• Same Ranking, Different Winner: How Scoring Targets Shape LLM Memory Benchmarks [2.9089118242427627]
  Conversational-memory systems transform dialogue history into facts, summaries, timelines, and other source-linked descendants.<n>This raises an underspecified evaluation question: which stored form should receive retrieval credit?<n>We show that this scoring-target choice is often left implicit and can materially change benchmark conclusions.
  arXiv  Detail & Related papers  (2026-05-22T02:53:07Z)
• Speak, Edit, Repeat: High-Fidelity Voice Editing and Zero-Shot TTS with Cross-Attentive Mamba [5.055749974859193]
  MAVE is a novel autoregressive architecture for text-conditioned voice editing and high-fidelity text-to-speech synthesis.<n>MAVE achieves state-of-the-art performance in speech editing and very competitive results in zero-shot TTS.<n>MAVE requires 6x less memory than VoiceCraft during inference on utterances from the RealEdit database.
  arXiv  Detail & Related papers  (2025-10-06T12:11:31Z)
• High Accuracy, Less Talk (HALT): Reliable LLMs through Capability-Aligned Finetuning [84.52940628494879]
  Large Language Models (LLMs) currently respond to every prompt.<n>LLMs can produce incorrect answers when they lack knowledge or capability.<n>We propose post-training an LLM to generate content only when confident in its correctness and to otherwise abstain.
  arXiv  Detail & Related papers  (2025-06-04T15:16:21Z)
• HEMA : A Hippocampus-Inspired Extended Memory Architecture for Long-Context AI Conversations [0.0]
  Large language models (LLMs) struggle with maintaining coherence in extended conversations spanning hundreds of turns.<n>This paper introduces HEMA, a dual-memory system inspired by human cognitive processes.<n>With 10K indexed chunks, Vector Memory achieves P@5 >= 0.80 and R@50 >= 0.74, doubling the area under the precision-recall curve.
  arXiv  Detail & Related papers  (2025-04-23T14:27:12Z)
• Error Correction by Paying Attention to Both Acoustic and Confidence References for Automatic Speech Recognition [52.624909026294105]
  We propose a non-autoregressive speech error correction method. A Confidence Module measures the uncertainty of each word of the N-best ASR hypotheses. The proposed system reduces the error rate by 21% compared with the ASR model.
  arXiv  Detail & Related papers  (2024-06-29T17:56:28Z)
• OpenSR: Open-Modality Speech Recognition via Maintaining Multi-Modality Alignment [57.15449072423539]
  We propose a training system Open-modality Speech Recognition (textbfOpenSR) OpenSR enables modality transfer from one to any in three different settings. It achieves highly competitive zero-shot performance compared to the existing few-shot and full-shot lip-reading methods.
  arXiv  Detail & Related papers  (2023-06-10T11:04:10Z)
• Wav2vec-Switch: Contrastive Learning from Original-noisy Speech Pairs for Robust Speech Recognition [52.71604809100364]
  We propose wav2vec-Switch, a method to encode noise robustness into contextualized representations of speech. Specifically, we feed original-noisy speech pairs simultaneously into the wav2vec 2.0 network. In addition to the existing contrastive learning task, we switch the quantized representations of the original and noisy speech as additional prediction targets.
  arXiv  Detail & Related papers  (2021-10-11T00:08:48Z)
• Unsupervised Speech Recognition [55.864459085947345]
  wav2vec-U, short for wav2vec Unsupervised, is a method to train speech recognition models without any labeled data. We leverage self-supervised speech representations to segment unlabeled audio and learn a mapping from these representations to phonemes via adversarial training. On the larger English Librispeech benchmark, wav2vec-U achieves a word error rate of 5.9 on test-other, rivaling some of the best published systems trained on 960 hours of labeled data from only two years ago.
  arXiv  Detail & Related papers  (2021-05-24T04:10:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.