Enhancing Flow Matching with A Unified Guidance Framework for Efficient and Robust Speech Synthesis

• URL: http://arxiv.org/abs/2607.00363v1
• Date: Wed, 01 Jul 2026 03:02:31 GMT
• Title: Enhancing Flow Matching with A Unified Guidance Framework for Efficient and Robust Speech Synthesis
• Abstract summary: Flow Matching (FM) has emerged as a powerful paradigm for speech generation but remains constrained by high inference and latency.<n>We propose a unified guidance framework that enhances generation efficiency and robustness through two complementary strategies.
• Score: 11.116469047991798
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Flow Matching (FM) has emerged as a powerful paradigm for speech generation but remains constrained by high inference latency and timbre leakage. To address these bottlenecks, we propose a unified guidance framework that enhances generation efficiency and robustness through two complementary strategies. On the data front, we introduce Data-guidance via heterogeneous augmentation, encouraging the model to disentangle linguistic content from acoustic residue. In parallel, we propose an enhanced Model-guidance mechanism that synergizes trajectory rectification with a novel intrinsic guidance objective. This approach distills conditional knowledge into network weights and straightens inference trajectory path, thereby eliminating Classifier-Free Guidance (CFG) overhead. Experiments demonstrate that our framework accelerates inference by nearly three times while effectively improving speaker similarity compared to state-of-the-art baselines.

Related papers

• AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters [67.51886052453452]
  Speculative decoding has become a prevalent paradigm for accelerating large language model inference.<n>Recent work such as DFlash further boosts drafting efficiency by leveraging diffusion drafters.<n>We propose AdaFlash framework, comprising two components: (i) an on-policy distillation (OPD) algorithm with reverse-KL divergence tailored for diffusion drafters, and (ii) an adaptive length head that dynamically adjusts the candidate sequence length on the fly.
  arXiv  Detail & Related papers  (2026-07-21T15:52:50Z)
• Self-Distilled Trajectory-Aware Boltzmann Modeling: Bridging the Training-Inference Discrepancy in Diffusion Language Models [65.89572755202245]
  Diffusion Language Models (DLMs) offer stronger global awareness and highly parallel generation.<n>Post-training DLMs with standard Negative Evidence Lower Bound (NELBO)-based supervised fine-tuning remains inefficient.<n>We propose a self-distilled trajectory-based post-training framework that aligns training with the easy-to-hard structure of inference.
  arXiv  Detail & Related papers  (2026-05-12T09:39:06Z)
• Rethinking Entropy Allocation in LLM-based ASR: Understanding the Dynamics between Speech Encoders and LLMs [17.167595029948576]
  We introduce three metrics to characterize how training paradigms allocate entropy reduction between the speech encoder and the LLM.<n>We propose a principled multi-stage training strategy grounded in capability-boundary awareness, optimizing parameter efficiency and robustness hallucination.<n> Experiments on Mandarin and English benchmarks show that our method achieves competitive performance with state-of-the-art models using only 2.3B parameters.
  arXiv  Detail & Related papers  (2026-04-09T09:07:52Z)
• TopoCurate:Modeling Interaction Topology for Tool-Use Agent Training [53.93696896939915]
  Training tool-use agents typically rely on Supervised Fine-Tuning (SFT) on successful trajectories and Reinforcement Learning (RL) on pass-rate-selected tasks.<n>We propose TopoCurate, an interaction-aware framework that projects multi-trial rollouts from the same task into a unified semantic quotient topology.<n>TopoCurate achieves consistent gains of 4.2% (SFT) and 6.9% (RL) over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-03-02T10:38:54Z)
• Euphonium: Steering Video Flow Matching via Process Reward Gradient Guided Stochastic Dynamics [49.242224984144904]
  We propose Euphonium, a novel framework that steers generation via process reward gradient guided dynamics.<n>Our key insight is to formulate the sampling process as a theoretically principled algorithm that explicitly incorporates the gradient of a Process Reward Model.<n>We derive a distillation objective that internalizes the guidance signal into the flow network, eliminating inference-time dependency on the reward model.
  arXiv  Detail & Related papers  (2026-02-04T08:59:57Z)
• Training-Free Representation Guidance for Diffusion Models with a Representation Alignment Projector [14.027059904924135]
  We introduce a representation alignment projector that injects representations predicted by a projector into intermediate sampling steps.<n>Experiments on SiTs and REPAs show notable improvements in class-conditional ImageNet synthesis.<n>The proposed method outperforms representative guidance when applied to SiT models.
  arXiv  Detail & Related papers  (2026-01-30T02:29:54Z)
• Beyond Confidence: Adaptive and Coherent Decoding for Diffusion Language Models [64.92045568376705]
  Coherent Contextual Decoding (CCD) is a novel inference framework built upon two core innovations.<n>CCD employs a trajectory rectification mechanism that leverages historical context to enhance sequence coherence.<n>Instead of rigid allocations based on diffusion steps, we introduce an adaptive sampling strategy that dynamically adjusts the unmasking budget for each step.
  arXiv  Detail & Related papers  (2025-11-26T09:49:48Z)
• Drax: Speech Recognition with Discrete Flow Matching [26.991421132974097]
  Diffusion and flow-based non-autoregressive (NAR) models have shown strong promise in large language modeling.<n>We propose Drax, a discrete flow matching framework for automatic speech recognition (ASR)<n>We construct an audio-conditioned probability path that guides the model through trajectories resembling likely intermediate inference errors.
  arXiv  Detail & Related papers  (2025-10-05T11:38:01Z)
• Latent Diffusion Model Based Denoising Receiver for 6G Semantic Communication: From Stochastic Differential Theory to Application [11.385703484113552]
  We propose a novel semantic communication framework empowered by generative artificial intelligence (GAI)<n>A latent diffusion model (LDM)-based semantic communication framework is proposed that combines a variational autoencoder for semantic features extraction.<n>The proposed system is a training-free framework that supports zero-shot generalization, and achieves superior performance under low-SNR and out-of-distribution conditions.
  arXiv  Detail & Related papers  (2025-06-06T03:20:32Z)
• E2ED^2:Direct Mapping from Noise to Data for Enhanced Diffusion Models [15.270657838960114]
  Diffusion models have established themselves as the de facto primary paradigm in visual generative modeling.<n>We present a novel end-to-end learning paradigm that establishes direct optimization from the final generated samples to initial noises.<n>Our method achieves substantial performance gains in terms of Fr'eche't Inception Distance (FID) and CLIP score, even with fewer sampling steps.
  arXiv  Detail & Related papers  (2024-12-30T16:06:31Z)
• Time-domain Speech Enhancement with Generative Adversarial Learning [53.74228907273269]
  This paper proposes a new framework called Time-domain Speech Enhancement Generative Adversarial Network (TSEGAN) TSEGAN is an extension of the generative adversarial network (GAN) in time-domain with metric evaluation to mitigate the scaling problem. In addition, we provide a new method based on objective function mapping for the theoretical analysis of the performance of Metric GAN.
  arXiv  Detail & Related papers  (2021-03-30T08:09:49Z)
• Automated and Formal Synthesis of Neural Barrier Certificates for Dynamical Models [70.70479436076238]
  We introduce an automated, formal, counterexample-based approach to synthesise Barrier Certificates (BC) The approach is underpinned by an inductive framework, which manipulates a candidate BC structured as a neural network, and a sound verifier, which either certifies the candidate's validity or generates counter-examples. The outcomes show that we can synthesise sound BCs up to two orders of magnitude faster, with in particular a stark speedup on the verification engine.
  arXiv  Detail & Related papers  (2020-07-07T07:39:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.