Mask2Flow-TSE: Two-Stage Target Speaker Extraction with Masking and Flow Matching

• URL: http://arxiv.org/abs/2603.12837v1
• Date: Fri, 13 Mar 2026 09:40:13 GMT
• Title: Mask2Flow-TSE: Two-Stage Target Speaker Extraction with Masking and Flow Matching
• Abstract summary: Target speaker extraction (TSE) extracts the target speaker's voice from overlapping speech mixtures given a reference utterance.<n>Discriminative methods apply time-frequency masking for fast inference but often over-suppress the target signal.<n>generative methods synthesize high-quality speech at the cost of numerous iterative steps.<n>We propose Mask2Flow-TSE, a two-stage framework combining the strengths of both paradigms.
• Score: 17.73633095059539
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Target speaker extraction (TSE) extracts the target speaker's voice from overlapping speech mixtures given a reference utterance. Existing approaches typically fall into two categories: discriminative and generative. Discriminative methods apply time-frequency masking for fast inference but often over-suppress the target signal, while generative methods synthesize high-quality speech at the cost of numerous iterative steps. We propose Mask2Flow-TSE, a two-stage framework combining the strengths of both paradigms. The first stage applies discriminative masking for coarse separation, and the second stage employs flow matching to refine the output toward target speech. Unlike generative approaches that synthesize speech from Gaussian noise, our method starts from the masked spectrogram, enabling high-quality reconstruction in a single inference step. Experiments show that Mask2Flow-TSE achieves comparable performance to existing generative TSE methods with approximately 85M parameters.

Related papers

• GenMask: Adapting DiT for Segmentation via Direct Mask Generation [81.54526445834294]
  We argue that instead of indirect adaptation, segmentation tasks should be trained directly in a generative manner.<n>We introduce timesteps sampling strategy for binary masks that emphasizes extreme noise levels for segmentation and moderate noise for image generation.<n>We present GenMask, a DiT trains to generate black-and-white segmentation masks as well as colorful images in RGB space under the original generative objective.
  arXiv  Detail & Related papers  (2026-03-25T03:52:05Z)
• GenTSE: Enhancing Target Speaker Extraction via a Coarse-to-Fine Generative Language Model [35.12859489567766]
  We present GenTSE, a two-stage decoder-only generative LM approach for TSE.<n> separating semantics and acoustics stabilizes decoding and yields more faithful, content-aligned target speech.<n>Experiments on Libri2Mix show that GenTSE surpasses previous LM-based systems in speech quality, intelligibility, and speaker consistency.
  arXiv  Detail & Related papers  (2025-12-24T06:13:02Z)
• FlowTSE: Target Speaker Extraction with Flow Matching [16.054014378418316]
  FlowTSE is a simple yet effective TSE approach based on conditional flow matching.<n>For tasks where phase reconstruction is crucial, we propose a novel vocoder conditioned on the complex STFT of the mixed signal.
  arXiv  Detail & Related papers  (2025-05-20T15:01:30Z)
• Qifusion-Net: Layer-adapted Stream/Non-stream Model for End-to-End Multi-Accent Speech Recognition [1.0690007351232649]
  We propose a layer-adapted fusion (LAF) model, called Qifusion-Net, which does not require any prior knowledge about the target accent. Experiment results demonstrate that our proposed methods outperform the baseline with relative reductions of 22.1$%$ and 17.2$%$ in character error rate (CER) across multi accent test datasets.
  arXiv  Detail & Related papers  (2024-07-03T11:35:52Z)
• High-Fidelity Speech Synthesis with Minimal Supervision: All Using Diffusion Models [56.00939852727501]
  Minimally-supervised speech synthesis decouples TTS by combining two types of discrete speech representations. Non-autoregressive framework enhances controllability, and duration diffusion model enables diversified prosodic expression.
  arXiv  Detail & Related papers  (2023-09-27T09:27:03Z)
• DDTSE: Discriminative Diffusion Model for Target Speech Extraction [62.422291953387955]
  We introduce the Discriminative Diffusion model for Target Speech Extraction (DDTSE) We apply the same forward process as diffusion models and utilize the reconstruction loss similar to discriminative methods. We devise a two-stage training strategy to emulate the inference process during model training.
  arXiv  Detail & Related papers  (2023-09-25T04:58:38Z)
• TranSpeech: Speech-to-Speech Translation With Bilateral Perturbation [61.564874831498145]
  TranSpeech is a speech-to-speech translation model with bilateral perturbation. We establish a non-autoregressive S2ST technique, which repeatedly masks and predicts unit choices. TranSpeech shows a significant improvement in inference latency, enabling speedup up to 21.4x than autoregressive technique.
  arXiv  Detail & Related papers  (2022-05-25T06:34:14Z)
• Speaker Embedding-aware Neural Diarization: a Novel Framework for Overlapped Speech Diarization in the Meeting Scenario [51.5031673695118]
  We reformulate overlapped speech diarization as a single-label prediction problem. We propose the speaker embedding-aware neural diarization (SEND) system.
  arXiv  Detail & Related papers  (2022-03-18T06:40:39Z)
• Speech segmentation using multilevel hybrid filters [0.0]
  A novel approach for speech segmentation is proposed, based on Multilevel Hybrid (mean/min) Filters (MHF) The proposed method is based on spectral changes, with the goal of segmenting the voice into homogeneous acoustic segments. This algorithm is being used for phoneticallysegmented speech coder, with successful results.
  arXiv  Detail & Related papers  (2022-02-24T00:03:02Z)
• Discretization and Re-synthesis: an alternative method to solve the Cocktail Party Problem [65.25725367771075]
  This study demonstrates, for the first time, that the synthesis-based approach can also perform well on this problem. Specifically, we propose a novel speech separation/enhancement model based on the recognition of discrete symbols. By utilizing the synthesis model with the input of discrete symbols, after the prediction of discrete symbol sequence, each target speech could be re-synthesized.
  arXiv  Detail & Related papers  (2021-12-17T08:35:40Z)
• Simultaneous Denoising and Dereverberation Using Deep Embedding Features [64.58693911070228]
  We propose a joint training method for simultaneous speech denoising and dereverberation using deep embedding features. At the denoising stage, the DC network is leveraged to extract noise-free deep embedding features. At the dereverberation stage, instead of using the unsupervised K-means clustering algorithm, another neural network is utilized to estimate the anechoic speech.
  arXiv  Detail & Related papers  (2020-04-06T06:34:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.