Achieving Timestamp Prediction While Recognizing with Non-Autoregressive End-to-End ASR Model

• URL: http://arxiv.org/abs/2301.12343v1
• Date: Sun, 29 Jan 2023 03:47:59 GMT
• Title: Achieving Timestamp Prediction While Recognizing with Non-Autoregressive End-to-End ASR Model
• Abstract summary: This paper proposes to perform timestamp prediction(TP) while recognizing by utilizing continuous integrate-and-fire(CIF) mechanism. Experiment results show that the proposed optimization methods significantly improve the accuracy of CIF timestamps.
• Score: 39.892106056198614
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Conventional ASR systems use frame-level phoneme posterior to conduct force-alignment~(FA) and provide timestamps, while end-to-end ASR systems especially AED based ones are short of such ability. This paper proposes to perform timestamp prediction~(TP) while recognizing by utilizing continuous integrate-and-fire~(CIF) mechanism in non-autoregressive ASR model - Paraformer. Foucing on the fire place bias issue of CIF, we conduct post-processing strategies including fire-delay and silence insertion. Besides, we propose to use scaled-CIF to smooth the weights of CIF output, which is proved beneficial for both ASR and TP task. Accumulated averaging shift~(AAS) and diarization error rate~(DER) are adopted to measure the quality of timestamps and we compare these metrics of proposed system and conventional hybrid force-alignment system. The experiment results over manually-marked timestamps testset show that the proposed optimization methods significantly improve the accuracy of CIF timestamps, reducing 66.7\% and 82.1\% of AAS and DER respectively. Comparing to Kaldi force-alignment trained with the same data, optimized CIF timestamps achieved 12.3\% relative AAS reduction.

Related papers

• REDDIT: Correcting Model-Generated Timestamp Drift in ASR without Forgetting via Replay-Based Distribution Editing [46.75114860324604]
  Autoregressive ASR systems can emit timestamps as decoded tokens, enabling timestamped transcription without frame-level or inference-time post-processing.<n>We show that these generated timestamps can drift across long non-speech spans.<n>We propose REDDIT(REplay-based Distribution eDITing), a lightweight two-stage post-training framework that corrects timestamps while avoiding forgetting.
  arXiv  Detail & Related papers  (2026-07-06T17:40:54Z)
• STEAM: Self-Supervised Temporal Ensemble Advantage Modeling for Real-World Robot Learning [76.76959348575764]
  Policy learning requires frame-level advantages that distinguish reliable local progress from failures and regressions.<n>We propose Self-supervised Temporal Temporal Ensemble Advantage Modeling (STEAM), a label-free method that learns such advantages from expert demonstrations.
  arXiv  Detail & Related papers  (2026-06-29T06:19:35Z)
• RAVEN: A Regime-Aware Variable-context Expert Network for Financial Time Series Forecasting [83.06074370551887]
  A fixed context window is mismatched to the time-varying optimal look-back of non-stationary price processes.<n>We propose a Mixture-of-Experts framework designed to adaptively determine the temporal context for each input sample.<n> Experiments on cumulative log-return prediction (HS300, S&P500) and fund sales forecasting demonstrate that RAVEN achieves SOTA performances, improves Pearson correlation by 9.2% on HS300 and 20.2% on S&P500, and reduces MSE by 18.2% on fund sales forecasting.
  arXiv  Detail & Related papers  (2026-06-23T02:11:04Z)
• VigilFormer: Deformable Attention for Video Anomaly Detection with Causal Risk Inference [2.538209532048867]
  We present a unified framework that combines deformable attention with causal temporal modeling to detect anomalies in video surveillance footage.<n>We evaluate our framework on VigilFormer, ShanghaiTech, and CUHK Avenue, at 41.5 FPS on a single GPU, recently outperforming weakly-supervised methods in both accuracy and speed.
  arXiv  Detail & Related papers  (2026-05-31T14:27:57Z)
• RareCP: Regime-Aware Retrieval for Efficient Conformal Prediction [11.259098206661077]
  We introduce RareCP, a regime-aware retrieval method for adaptive conformal time series prediction.<n>RareCP learns local calibration representations through a mixture of cosine-attention experts.<n>It forms a weighted conformal quantile over their signed residuals, yielding asymmetric prediction intervals.
  arXiv  Detail & Related papers  (2026-05-09T10:12:07Z)
• Efficient Punctuation Restoration via Weighted Lookahead Scoring Method for Streaming ASR Systems [4.703098868120046]
  Punctuation restoration improves ASR (Automatic Speech Recognition) readability.<n>Streaming ASR requires online decisions with limited future context.<n>This paper proposes a non-autoregressive scoring method that preserves the input transcript and makes a decision at each word boundary.
  arXiv  Detail & Related papers  (2026-04-18T01:03:33Z)
• Low-Dimensional Adaptation of Rectified Flow: A New Perspective through the Lens of Diffusion and Stochastic Localization [59.04314685837778]
  Rectified flow (RF) has gained considerable popularity due to its generation efficiency and state-of-the-art performance.<n>In this paper, we investigate the degree to which RF automatically adapts to the intrinsic low dimensionality of the support of the target distribution to accelerate sampling.<n>We show that, using a carefully designed choice of the time-discretization scheme and with sufficiently accurate drift estimates, the RF sampler enjoys an complexity of order $O(k/varepsilon)$.
  arXiv  Detail & Related papers  (2026-01-21T22:09:27Z)
• Enhancing Speech Emotion Recognition Leveraging Aligning Timestamps of ASR Transcripts and Speaker Diarization [4.1088673993841685]
  This paper investigates the impact of incorporating timestamp-based alignment between Automatic Speech Recognition (ASR) transcripts and Speaker Diarization (SD) outputs on Speech Emotion Recognition (SER) accuracy.<n>We introduce an alignment pipeline utilizing pre-trained ASR and speaker diarization models, systematically synchronizing timestamps to generate accurately labeled speaker segments.
  arXiv  Detail & Related papers  (2025-07-25T15:05:20Z)
• Breaking Determinism: Fuzzy Modeling of Sequential Recommendation Using Discrete State Space Diffusion Model [66.91323540178739]
  Sequential recommendation (SR) aims to predict items that users may be interested in based on their historical behavior. We revisit SR from a novel information-theoretic perspective and find that sequential modeling methods fail to adequately capture randomness and unpredictability of user behavior. Inspired by fuzzy information processing theory, this paper introduces the fuzzy sets of interaction sequences to overcome the limitations and better capture the evolution of users' real interests.
  arXiv  Detail & Related papers  (2024-10-31T14:52:01Z)
• Automatic Speech Recognition System-Independent Word Error Rate Estimation [23.25173244408922]
  Word error rate (WER) is a metric used to evaluate the quality of transcriptions produced by Automatic Speech Recognition (ASR) systems. In this paper, a hypothesis generation method for ASR System-Independent WER estimation is proposed.
  arXiv  Detail & Related papers  (2024-04-25T16:57:05Z)
• On the Relevance of Phoneme Duration Variability of Synthesized Training Data for Automatic Speech Recognition [0.552480439325792]
  We focus on the temporal structure of synthetic data and its relation to ASR training. We show how much the degradation of synthetic data quality is influenced by duration modeling in non-autoregressive TTS. Using a simple algorithm we shift phoneme duration distributions of the TTS system closer to real durations.
  arXiv  Detail & Related papers  (2023-10-12T08:45:21Z)
• Iterative pseudo-forced alignment by acoustic CTC loss for self-supervised ASR domain adaptation [80.12316877964558]
  High-quality data labeling from specific domains is costly and human time-consuming. We propose a self-supervised domain adaptation method, based upon an iterative pseudo-forced alignment algorithm.
  arXiv  Detail & Related papers  (2022-10-27T07:23:08Z)
• Error Correction in ASR using Sequence-to-Sequence Models [32.41875780785648]
  Post-editing in Automatic Speech Recognition entails automatically correcting common and systematic errors produced by the ASR system. We propose to use a powerful pre-trained sequence-to-sequence model, BART, to serve as a denoising model. Experimental results on accented speech data demonstrate that our strategy effectively rectifies a significant number of ASR errors.
  arXiv  Detail & Related papers  (2022-02-02T17:32:59Z)
• FastCorrect: Fast Error Correction with Edit Alignment for Automatic Speech Recognition [90.34177266618143]
  We propose FastCorrect, a novel NAR error correction model based on edit alignment. FastCorrect speeds up the inference by 6-9 times and maintains the accuracy (8-14% WER reduction) compared with the autoregressive correction model. It outperforms the accuracy of popular NAR models adopted in neural machine translation by a large margin.
  arXiv  Detail & Related papers  (2021-05-09T05:35:36Z)
• Reducing Streaming ASR Model Delay with Self Alignment [20.61461084287351]
  Constrained alignment is a well-known existing approach that penalizes predicted word boundaries using external low-latency acoustic models. FastEmit is a sequence-level delay regularization scheme encouraging vocabulary tokens over blanks without any reference alignments. In this paper, we propose a novel delay constraining method, named self alignment.
  arXiv  Detail & Related papers  (2021-05-06T18:00:11Z)
• Higher Performance Visual Tracking with Dual-Modal Localization [106.91097443275035]
  Visual Object Tracking (VOT) has synchronous needs for both robustness and accuracy. We propose a dual-modal framework for target localization, consisting of robust localization suppressingors via ONR and the accurate localization attending to the target center precisely via OFC.
  arXiv  Detail & Related papers  (2021-03-18T08:47:56Z)
• Change Point Detection in Time Series Data using Autoencoders with a Time-Invariant Representation [69.34035527763916]
  Change point detection (CPD) aims to locate abrupt property changes in time series data. Recent CPD methods demonstrated the potential of using deep learning techniques, but often lack the ability to identify more subtle changes in the autocorrelation statistics of the signal. We employ an autoencoder-based methodology with a novel loss function, through which the used autoencoders learn a partially time-invariant representation that is tailored for CPD.
  arXiv  Detail & Related papers  (2020-08-21T15:03:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.