AudioNoisePrints: Model-free audio watermarking using spatial correlation in flow matching TTS

• URL: http://arxiv.org/abs/2608.22186v1
• Date: Sun, 23 Aug 2026 02:52:37 GMT
• Title: AudioNoisePrints: Model-free audio watermarking using spatial correlation in flow matching TTS
• Abstract summary: We present AudioNoisePrints, a training-free watermarking pipeline for flow matching and diffusion TTS models.<n>We exploited the fact that there are strong correlations between the initial Gaussian noises and the generated outputs in diffusion and flow matching models.<n>Our method outperforms AudioSeal, a strong baseline for audio watermarking under strong augmentations.
• Score: 11.455535872504749
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present AudioNoisePrints, a training-free watermarking pipeline for flow matching and diffusion TTS models, which requires minimal extra computation during inference and does not require retraining the TTS model or reducing the generation quality. We exploited the fact that there are strong correlations between the initial Gaussian noises and the generated outputs in diffusion and flow matching models, such that a simple cosine correlation between the initial noise and the generated output can be used to perform watermaking. Moreover, we train a lightweight detector on top for more aggressive augmentations. Our method outperforms AudioSeal, a strong baseline for audio watermarking under strong augmentations. We experimented on F5TTS and other TTS and vocoder models, and concluded that they all exhibit similar spatial correlation properties, suggesting our watermarking scheme can be used for more flow-matching TTS models and even vocoders in the future.

Related papers

• WavFlow: Audio Generation in Waveform Space [25.62432990858879]
  WavFlow is a framework that generates high-fidelity audio directly in raw waveform space without intermediate representations.<n>We leverage an automated data pipeline to curate 5 million high-quality video-text-audio triplets.<n>WavFlow achieves competitive performance on the video-to-audio benchmark VGGSound (FD_PaSST: 59.98, IS_PANNs: 17.40, DeSync: 0.44) and the text-to-audio benchmark AudioCaps (FD_PANNs: 10.63, IS_PANNs: 12.62)
  arXiv  Detail & Related papers  (2026-05-18T17:59:10Z)
• Talker-T2AV: Joint Talking Audio-Video Generation with Autoregressive Diffusion Modeling [59.97930201616015]
  We propose Talker-T2AV, an autoregressive diffusion framework where high-level cross-modal modeling occurs in a shared backbone.<n>A shared autoregressive language model jointly reasons over audio and video in a unified patch-level token space.<n> Experiments on talking portrait benchmarks show Talker-T2AV outperforms dual-branch baselines in lip-sync accuracy, video quality, and audio quality.
  arXiv  Detail & Related papers  (2026-04-26T07:48:47Z)
• DiT-Flow: Speech Enhancement Robust to Multiple Distortions based on Flow Matching in Latent Space and Diffusion Transformers [22.00618863958246]
  DiT-Flow is a flow matching-based SE framework built on the latent Diffusion Transformer (DiT) backbone.<n>We show that DiT-Flow consistently outperforms state-of-the-art generative SE models.
  arXiv  Detail & Related papers  (2026-03-23T06:03:58Z)
• Smark: A Watermark for Text-to-Speech Diffusion Models via Discrete Wavelet Transform [7.904062289123103]
  This paper proposes a universal watermarking scheme for TTS diffusion models, termed Smark.<n>Smark operates in the common reverse diffusion paradigm shared by all TTS diffusion models.<n>Experiments are conducted to evaluate the audio quality and watermark performance in various simulated real-world attack scenarios.
  arXiv  Detail & Related papers  (2025-12-21T16:07:08Z)
• T2SMark: Balancing Robustness and Diversity in Noise-as-Watermark for Diffusion Models [89.29541056113442]
  T2SMark is a two-stage watermarking scheme based on Tail-Truncated Sampling (TTS)<n>We evaluate T2SMark on diffusion models with both U-Net and DiT backbones.
  arXiv  Detail & Related papers  (2025-10-25T16:55:55Z)
• Next Tokens Denoising for Speech Synthesis [51.320443764269726]
  Dragon-FM is a novel text-to-speech (TTS) design that unifies AR and flow-matching.<n>It processes 48 kHz audio tokens in chunks at a compact rate of 12.5 tokens per second.<n>Experiments on podcast datasets demonstrate its capability to efficiently generate high-quality zero-shot podcasts.
  arXiv  Detail & Related papers  (2025-07-30T15:03:36Z)
• AudioTurbo: Fast Text-to-Audio Generation with Rectified Diffusion [23.250409921931492]
  Rectified flow enhances inference speed by learning straight-line ordinary differential equation paths.<n>This approach requires training a flow-matching model from scratch and tends to perform suboptimally, or even poorly, at low step counts.<n>We propose AudioTurbo, which learns first-order ODE paths from deterministic noise sample pairs generated by a pre-trained TTA model.
  arXiv  Detail & Related papers  (2025-05-28T08:33:58Z)
• Diffusion-Driven Semantic Communication for Generative Models with Bandwidth Constraints [66.63250537475973]
  This paper introduces a diffusion-driven semantic communication framework with advanced VAE-based compression for bandwidth-constrained generative model.<n>Our experimental results demonstrate significant improvements in pixel-level metrics like peak signal to noise ratio (PSNR) and semantic metrics like learned perceptual image patch similarity (LPIPS)
  arXiv  Detail & Related papers  (2024-07-26T02:34:25Z)
• Frieren: Efficient Video-to-Audio Generation Network with Rectified Flow Matching [51.70360630470263]
  Video-to-audio (V2A) generation aims to synthesize content-matching audio from silent video.<n>We propose Frieren, a V2A model based on rectified flow matching.<n>Experiments indicate that Frieren achieves state-of-the-art performance in both generation quality and temporal alignment.
  arXiv  Detail & Related papers  (2024-06-01T06:40:22Z)
• ConsistencyTTA: Accelerating Diffusion-Based Text-to-Audio Generation with Consistency Distillation [21.335983674309475]
  Diffusion models suffer from slow inference due to an excessive number of queries to the underlying denoising network per generation. We introduce ConsistencyTTA, a framework requiring only a single non-autoregressive network query. We achieve so by proposing "CFG-aware latent consistency model," which adapts consistency generation into a latent space.
  arXiv  Detail & Related papers  (2023-09-19T16:36:33Z)
• From Discrete Tokens to High-Fidelity Audio Using Multi-Band Diffusion [84.138804145918]
  Deep generative models can generate high-fidelity audio conditioned on various types of representations. These models are prone to generate audible artifacts when the conditioning is flawed or imperfect. We propose a high-fidelity multi-band diffusion-based framework that generates any type of audio modality from low-bitrate discrete representations.
  arXiv  Detail & Related papers  (2023-08-02T22:14:29Z)
• OverFlow: Putting flows on top of neural transducers for better TTS [9.346907121576258]
  Neural HMMs are a type of neural transducer recently proposed for sequence-to-sequence modelling in text-to-speech. In this paper, we combine neural HMM TTS with normalising flows for describing the highly non-Gaussian distribution of speech acoustics.
  arXiv  Detail & Related papers  (2022-11-13T12:53:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.