Woosh: A Sound Effects Foundation Model

• URL: http://arxiv.org/abs/2604.01929v1
• Date: Thu, 02 Apr 2026 11:49:00 GMT
• Title: Woosh: A Sound Effects Foundation Model
• Abstract summary: We present Woosh, Sony AI's publicly released sound effect foundation model.<n>Being optimized for sound effects, we provide (1) a high-quality audio encoder/decoder model and (2) a text-audio alignment model for conditioning.<n>Distilled text-to-audio and video-to-audio models are also included in the release, allowing for low-resource operation and fast inference.
• Score: 58.759449707459474
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The audio research community depends on open generative models as foundational tools for building novel approaches and establishing baselines. In this report, we present Woosh, Sony AI's publicly released sound effect foundation model, detailing its architecture, training process, and an evaluation against other popular open models. Being optimized for sound effects, we provide (1) a high-quality audio encoder/decoder model and (2) a text-audio alignment model for conditioning, together with (3) text-to-audio and (4) video-to-audio generative models. Distilled text-to-audio and video-to-audio models are also included in the release, allowing for low-resource operation and fast inference. Our evaluation on both public and private data shows competitive or better performance for each module when compared to existing open alternatives like StableAudio-Open and TangoFlux. Inference code and model weights are available at https://github.com/SonyResearch/Woosh. Demo samples can be found at https://sonyresearch.github.io/Woosh/.

Related papers

• MOVA: Towards Scalable and Synchronized Video-Audio Generation [91.56945636522345]
  We introduce MOVA (MOSS Video and Audio), an open-source model capable of generating high-quality, synchronized audio-visual content.<n>By releasing the model weights and code, we aim to advance research and foster a vibrant community of creators.
  arXiv  Detail & Related papers  (2026-02-09T15:31:54Z)
• MMAudioSep: Taming Video-to-Audio Generative Model Towards Video/Text-Queried Sound Separation [34.79792511587843]
  We introduce MMAudioSep, a generative model for video/text-queried sound separation.<n>By leveraging knowledge about the relationship between video/text and audio learned through a pretrained audio generative model, we can train the model more efficiently.
  arXiv  Detail & Related papers  (2025-10-10T07:13:06Z)
• Can Synthetic Audio From Generative Foundation Models Assist Audio Recognition and Speech Modeling? [40.3708221702947]
  We aim to evaluate the quality of audio generation by examining the effectiveness of using them as training data. Specifically, we conduct studies to explore the use of synthetic audio for audio recognition. We also investigate whether synthetic audio can serve as a resource for data augmentation in speech-related modeling.
  arXiv  Detail & Related papers  (2024-06-13T04:33:05Z)
• Tango 2: Aligning Diffusion-based Text-to-Audio Generations through Direct Preference Optimization [70.13218512896032]
  Generation of audio from text prompts is an important aspect of such processes in the music and film industry. Our hypothesis is focusing on how these aspects of audio generation could improve audio generation performance in the presence of limited data. We synthetically create a preference dataset where each prompt has a winner audio output and some loser audio outputs for the diffusion model to learn from.
  arXiv  Detail & Related papers  (2024-04-15T17:31:22Z)
• Audiobox: Unified Audio Generation with Natural Language Prompts [37.39834044113061]
  This paper presents Audiobox, a unified model based on flow-matching that is capable of generating various audio modalities. We design description-based and example-based prompting to enhance controllability and unify speech and sound generation paradigms. Audiobox sets new benchmarks on speech and sound generation and unlocks new methods for generating audio with novel vocal and acoustic styles.
  arXiv  Detail & Related papers  (2023-12-25T22:24:49Z)
• AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining [46.22290575167155]
  This paper proposes a framework that utilizes the same learning method for speech, music, and sound effect generation. Our framework introduces a general representation of audio, called "language of audio" (LOA)
  arXiv  Detail & Related papers  (2023-08-10T17:55:13Z)
• Large-scale unsupervised audio pre-training for video-to-speech synthesis [64.86087257004883]
  Video-to-speech synthesis is the task of reconstructing the speech signal from a silent video of a speaker. In this paper we propose to train encoder-decoder models on more than 3,500 hours of audio data at 24kHz. We then use the pre-trained decoders to initialize the audio decoders for the video-to-speech synthesis task.
  arXiv  Detail & Related papers  (2023-06-27T13:31:33Z)
• Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models [65.18102159618631]
  multimodal generative modeling has created milestones in text-to-image and text-to-video generation. Its application to audio still lags behind for two main reasons: the lack of large-scale datasets with high-quality text-audio pairs, and the complexity of modeling long continuous audio data. We propose Make-An-Audio with a prompt-enhanced diffusion model that addresses these gaps.
  arXiv  Detail & Related papers  (2023-01-30T04:44:34Z)
• Generating Visually Aligned Sound from Videos [83.89485254543888]
  We focus on the task of generating sound from natural videos. The sound should be both temporally and content-wise aligned with visual signals. Some sounds generated outside of a camera can not be inferred from video content.
  arXiv  Detail & Related papers  (2020-07-14T07:51:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.