REDDIT: Correcting Model-Generated Timestamp Drift in ASR without Forgetting via Replay-Based Distribution Editing

• URL: http://arxiv.org/abs/2607.05364v2
• Date: Wed, 15 Jul 2026 17:16:51 GMT
• Title: REDDIT: Correcting Model-Generated Timestamp Drift in ASR without Forgetting via Replay-Based Distribution Editing
• Abstract summary: Autoregressive ASR systems can emit timestamps as decoded tokens, enabling timestamped transcription without frame-level or inference-time post-processing.<n>We show that these generated timestamps can drift across long non-speech spans.<n>We propose REDDIT(REplay-based Distribution eDITing), a lightweight two-stage post-training framework that corrects timestamps while avoiding forgetting.
• Score: 46.75114860324604
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Modern autoregressive ASR systems can emit timestamps as decoded tokens, enabling timestamped transcription without frame-level aligners or inference-time post-processing. We show that these generated timestamps can drift across long non-speech spans: the transcript may remain plausible, but the decoded time axis drifts away from the audio. We study this non-speech-induced timestamp drift with self-built gap and long-gap benchmarks across 15 evaluated timestamp-producing ASR and audio-language systems. Naive timestamp-corrected fine-tuning improves alignment but can severely degrade non-target ASR behavior, exposing a forgetting problem. We propose REDDIT(REplay-based Distribution eDITing), a lightweight two-stage post-training framework that corrects timestamps while avoiding this catastrophic forgetting: it first edits timestamp targets under the model's own replayed decoder context while matching the frozen base distribution on non-timestamp tokens, then applies a short edited-prefix refinement stage. In this framework, we construct correction supervision without human transcripts or human timestamp annotations by combining VAD-trimmed speech spans with inserted non-speech gaps and known concatenation offsets. On Whisper-tiny, 34.9 hours of targeted correction audio used and only 1.6% of model parameters updated, raising long-gap mIoU from 38.7% to 95.0% and reducing mixed-gap out-of-domain AAS from 2752 ms to 223 ms while preserving CV-en MER at 41.3% (versus 524.2% for ordinary SFT decoder tuning).

Related papers

• Topic-to-Timestamp Alignment by Constrained Evidence Selection [5.582495870293832]
  Given a natural-language topic and a timestamped meeting transcript, the goal is to return the time at which the topic is discussed.<n>A standard RAG setup can retrieve relevant transcript excerpts, but still asks the language model to generate a timestamp.<n>We recast timestamp prediction as constrained temporal candidate selection.
  arXiv  Detail & Related papers  (2026-06-18T19:38:57Z)
• In-Sync: Adaptation of Speech Aware Large Language Models for ASR with Word Level Timestamp Predictions [51.236148875603185]
  We extend an existing speech-aware language model to predict timestamps directly alongside transcripts.<n>We introduce a set of novel lightweight training strategies that improve alignment robustness while preserving recognition quality.
  arXiv  Detail & Related papers  (2026-04-14T20:56:24Z)
• TimeStampEval: A Simple LLM Eval and a Little Fuzzy Matching Trick to Improve Search Accuracy [0.0]
  TimeStampEval is a benchmark for retrieving precise millisecond timestamps from long transcripts given non-verbatim quotes.<n>Our simple two-stage method dramatically improves retrieval accuracy while cutting inference costs by over 90%.<n>The motivating use case is an automated long-form podcast that assembles Congressional Record clips into AI-hosted verbatim narration.
  arXiv  Detail & Related papers  (2025-10-27T21:54:56Z)
• Word Level Timestamp Generation for Automatic Speech Recognition and Translation [28.176210372699618]
  We introduce a data-driven approach for enabling word-level timestamp prediction in the Canary model.<n>Our method demonstrates precision and recall rates between 80% and 90%, with timestamp prediction errors ranging from 20 to 120 ms across four languages.
  arXiv  Detail & Related papers  (2025-05-21T15:24:29Z)
• Timestamped Embedding-Matching Acoustic-to-Word CTC ASR [2.842794675894731]
  We describe a novel method of training an embedding-matching word-level connectionist temporal classification (CTC) automatic speech recognizer (ASR) The word timestamps enable the ASR to output word segmentations and word confusion networks without relying on a secondary model or forced alignment process when testing.
  arXiv  Detail & Related papers  (2023-06-20T11:53:43Z)
• Achieving Timestamp Prediction While Recognizing with Non-Autoregressive End-to-End ASR Model [39.892106056198614]
  This paper proposes to perform timestamp prediction(TP) while recognizing by utilizing continuous integrate-and-fire(CIF) mechanism. Experiment results show that the proposed optimization methods significantly improve the accuracy of CIF timestamps.
  arXiv  Detail & Related papers  (2023-01-29T03:47:59Z)
• Iterative pseudo-forced alignment by acoustic CTC loss for self-supervised ASR domain adaptation [80.12316877964558]
  High-quality data labeling from specific domains is costly and human time-consuming. We propose a self-supervised domain adaptation method, based upon an iterative pseudo-forced alignment algorithm.
  arXiv  Detail & Related papers  (2022-10-27T07:23:08Z)
• FastCorrect: Fast Error Correction with Edit Alignment for Automatic Speech Recognition [90.34177266618143]
  We propose FastCorrect, a novel NAR error correction model based on edit alignment. FastCorrect speeds up the inference by 6-9 times and maintains the accuracy (8-14% WER reduction) compared with the autoregressive correction model. It outperforms the accuracy of popular NAR models adopted in neural machine translation by a large margin.
  arXiv  Detail & Related papers  (2021-05-09T05:35:36Z)
• Advanced Long-context End-to-end Speech Recognition Using Context-expanded Transformers [56.56220390953412]
  We extend our prior work by introducing the Conformer architecture to further improve the accuracy. We demonstrate that the extended Transformer provides state-of-the-art end-to-end ASR performance.
  arXiv  Detail & Related papers  (2021-04-19T16:18:00Z)
• FastEmit: Low-latency Streaming ASR with Sequence-level Emission Regularization [78.46088089185156]
  Streaming automatic speech recognition (ASR) aims to emit each hypothesized word as quickly and accurately as possible. Existing approaches penalize emission delay by manipulating per-token or per-frame probability prediction in sequence transducer models. We propose a sequence-level emission regularization method, named FastEmit, that applies latency regularization directly on per-sequence probability in training transducer models.
  arXiv  Detail & Related papers  (2020-10-21T17:05:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.