AST: Adaptive, Seamless, and Training-Free Precise Speech Editing

• URL: http://arxiv.org/abs/2604.16056v1
• Date: Fri, 17 Apr 2026 13:30:59 GMT
• Title: AST: Adaptive, Seamless, and Training-Free Precise Speech Editing
• Abstract summary: AST is an Adaptive, Seamless, and Training-free precise speech editing framework.<n>We show that AST improves consistency while reducing Word Error Rate by nearly 70%.<n>Applying AST to a foundation TTS model reduces WDTW by 27%, achieving state-of-the-art speaker preservation and temporal fidelity.
• Score: 28.699272599440032
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Text-based speech editing aims to modify specific segments while preserving speaker identity and acoustic context. Existing methods rely on task-specific training, which incurs high data costs and struggles with temporal fidelity in unedited regions. Meanwhile, adapting Text-to-Speech (TTS) models often faces a trade-off between editing quality and consistency. To address these issues, we propose AST, an Adaptive, Seamless, and Training-free precise speech editing framework. Leveraging a pre-trained autoregressive TTS model, AST introduces Latent Recomposition to selectively stitch preserved source segments with newly synthesized targets. Furthermore, AST extends this latent manipulation to enable precise style editing for specific speech segments. To prevent artifacts at these edit boundaries, the framework incorporates Adaptive Weak Fact Guidance (AWFG). AWFG dynamically modulates a mel-space guidance signal, enforcing structural constraints only where necessary without disrupting the generative manifold. To fill the gap of publicly accessible benchmarks, we introduce LibriSpeech-Edit, a new and larger speech editing dataset. As existing metrics poorly evaluate temporal consistency in unedited regions, we propose Word-level Dynamic Time Warping (WDTW). Extensive experiments demonstrate that AST resolves the controllability-quality trade-off without extra training. Compared to the previous most temporally consistent baseline, AST improves consistency while reducing Word Error Rate by nearly 70%. Moreover, applying AST to a foundation TTS model reduces WDTW by 27%, achieving state-of-the-art speaker preservation and temporal fidelity.

Related papers

• AuK Technical Report: An Open-Source Foundational Model for Speech Generation and Editing [74.97664372105693]
  We introduce AuK, an open-source foundational model that unifies speech generation and editing through a common interface of natural-language instructions and audio context.<n>We construct approximately 3.03 billion instruction--audio instances and 1.95 million hours of effective supervision across five task families.
  arXiv  Detail & Related papers  (2026-09-08T15:59:16Z)
• Test-time adaptation for speech enhancement with an autoregressive speech prior [40.592330125629644]
  Test-time adaptation (TTA) offers a promising direction for improving speech enhancement models under mismatched acoustic conditions.<n>We propose a single-utterance TTA method that regularizes a pretrained speech enhancement model using an autoregressive prior trained on clean speech latent representations extracted from a neural audio.
  arXiv  Detail & Related papers  (2026-09-03T10:10:23Z)
• Spatial Temporal Synergy: Balancing Change and Invariance in Text Driven 3D Human Motion Editing [78.44034844227868]
  Text-driven human motion editing aims to modify existing motion sequences according to natural language instructions.<n>Change and Invariance Motion Editing (CIME) is a unified framework that comprehensively decouples change and invariance into spatial pose and temporal rhythm dimensions.<n>Experiments on the MotionFix and STANCE Adjustment datasets demonstrate that CIME achieves state-of-the-art performance in editing alignment and structural fidelity.
  arXiv  Detail & Related papers  (2026-08-17T01:54:44Z)
• Temporal Contrastive Decoding: A Training-Free Method for Large Audio-Language Models [56.91801348360746]
  Large audio-language models (LALMs) generalize across speech, sound, and music.<n> unified decoders can exhibit a emphtemporal smoothing bias<n>We propose emphTemporal Contrastive Decoding (TCD), a training-free decoding method for unified LALMs.
  arXiv  Detail & Related papers  (2026-04-16T02:30:41Z)
• Bootstrapping Video Semantic Segmentation Model via Distillation-assisted Test-Time Adaptation [58.40817037271021]
  DiTTA (Distillation-assisted Test-Time Adaptation) is a novel framework that converts an ISS model into a temporally-aware VSS model without annotated videos.<n>We demonstrate DiTTA's effectiveness, achieving competitive or superior performance relative to fully-supervised VSS methods.
  arXiv  Detail & Related papers  (2026-04-13T03:47:08Z)
• Adaptive Duration Model for Text Speech Alignment [2.594813802197567]
  Speech-to-text alignment is a critical component of neural text to speech (TTS) models.<n>We propose a novel duration prediction framework that can give promising phoneme-level duration distribution with given text.
  arXiv  Detail & Related papers  (2025-07-30T12:31:11Z)
• Orthogonal Projection Subspace to Aggregate Online Prior-knowledge for Continual Test-time Adaptation [67.80294336559574]
  Continual Test Time Adaptation (CTTA) is a task that requires a source pre-trained model to continually adapt to new scenarios.<n>We propose a novel pipeline, Orthogonal Projection Subspace to aggregate online Prior-knowledge, dubbed OoPk.
  arXiv  Detail & Related papers  (2025-06-23T18:17:39Z)
• SUTA-LM: Bridging Test-Time Adaptation and Language Model Rescoring for Robust ASR [58.31068047426522]
  Test-Time Adaptation (TTA) aims to mitigate by adjusting models during inference.<n>Recent work explores combining TTA with external language models, using techniques like beam search rescoring or generative error correction.<n>We propose SUTA-LM, a simple yet effective extension of SUTA, with language model rescoring.<n> Experiments on 18 diverse ASR datasets show that SUTA-LM achieves robust results across a wide range of domains.
  arXiv  Detail & Related papers  (2025-06-10T02:50:20Z)
• Counterfactual Activation Editing for Post-hoc Prosody and Mispronunciation Correction in TTS Models [19.852233854729235]
  Existing approaches for prosody manipulation often depend on specialized modules or additional training, limiting their capacity for post-hoc adjustments.<n>We introduce Counterfactual Activation Editing, a model-agnostic method that manipulates internal representations in a pre-trained TTS model to achieve post-hoc control of prosody and pronunciation.<n> Experimental results show that our method effectively adjusts prosodic features and corrects mispronunciations while preserving synthesis quality.
  arXiv  Detail & Related papers  (2025-06-01T04:33:37Z)
• MegaTTS 3: Sparse Alignment Enhanced Latent Diffusion Transformer for Zero-Shot Speech Synthesis [56.25862714128288]
  This paper introduces textitMegaTTS 3, a zero-shot text-to-speech (TTS) system featuring an innovative sparse alignment algorithm.<n>Specifically, we provide sparse alignment boundaries to MegaTTS 3 to reduce the difficulty of alignment without limiting the search space.<n>Experiments demonstrate that MegaTTS 3 achieves state-of-the-art zero-shot TTS speech quality and supports highly flexible control over accent intensity.
  arXiv  Detail & Related papers  (2025-02-26T08:22:00Z)
• FluentEditor2: Text-based Speech Editing by Modeling Multi-Scale Acoustic and Prosody Consistency [40.95700389032375]
  Text-based speech editing (TSE) allows users to edit speech by modifying the corresponding text directly without altering the original recording.<n>Current TSE techniques often focus on minimizing discrepancies between generated speech and reference within edited regions during training to achieve fluent TSE performance.<n>We propose a new fluency speech editing scheme based on our previous textitFluentEditor model, termed textittextbfFluentEditor2.
  arXiv  Detail & Related papers  (2024-09-28T10:18:35Z)
• DiffEditor: Enhancing Speech Editing with Semantic Enrichment and Acoustic Consistency [20.3466261946094]
  We introduce DiffEditor, a novel speech editing model designed to enhance performance in OOD text scenarios. We enrich the semantic information of phoneme embeddings by integrating word embeddings extracted from a pretrained language model. We propose a first-order loss function to promote smoother transitions at editing boundaries and enhance the overall fluency of the edited speech.
  arXiv  Detail & Related papers  (2024-09-19T07:11:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.