Quality-Diversity Search in Sound Generation: Investigating Innovation Engines for Audio Exploration

• URL: http://arxiv.org/abs/2606.09780v1
• Date: Mon, 08 Jun 2026 17:40:09 GMT
• Title: Quality-Diversity Search in Sound Generation: Investigating Innovation Engines for Audio Exploration
• Abstract summary: This study addresses the challenges composers and sound designers face in creating and refining tools to achieve their musical goals.<n>We use evolutionary processes to promote diversity and foster serendipitous discoveries, arguing that diversity-promoting algorithms can bridge the gap between the theoretical realisation and practical accessibility of sounds.
• Score: 1.7411855207380258
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This study addresses the challenges composers and sound designers face in creating and refining tools to achieve their musical goals. Using evolutionary processes to promote diversity and foster serendipitous discoveries, we automate the search through uncharted sonic spaces for sound discovery, arguing that diversity-promoting algorithms can bridge the gap between the theoretical realisation and practical accessibility of sounds. We describe a system for generative sound synthesis combining Quality Diversity (QD) algorithms with a supervised discriminative model, inspired by the Innovation Engine algorithm, and explore different configurations and the interplay between the chosen synthesis approach and the discriminative model. We examine the interaction between Compositional Pattern Producing Networks (CPPNs) and Digital Signal Processing (DSP) graphs, introducing a novel approach that uses multiple specialised CPPNs for different frequency ranges; this yields simpler networks while maintaining performance comparable to single-CPPN setups. We also investigate evolutionary stepping stones by analysing goal switches between musical and non-musical contexts, revealing how lineages traverse unlikely paths to current elites. Expanding the behaviour space of a previous study to include various sound durations, we uncover specialisation within temporal niches. Results indicate that CPPN and DSP graphs coupled with a Multi-dimensional Archive of Phenotypic Elites (MAP-Elites) and a deep learning classifier can generate a substantial variety of synthetic sounds, diverse and innovative across temporal and contextual dimensions. We present the generated sound objects through an online explorer and as rendered sound files, and, in the context of music composition, an experimental application that showcases their creative potential across various durations and contexts.

Related papers

• AI-Based Sound Effect Generation: A Narrative Review of Generative Models Across Input Modalities [0.0]
  This chapter reviews and analyzes recent AI-based generative models for sound effect synthesis.<n>It examines 30 peer-reviewed articles sourced from Google Scholar, IEEE Xplore, and the ACM Digital Library.<n>The results show that multiple models achieved state-of-the-art performance, producing high-fidelity, semantically aligned, and increasingly temporally coherent sound effects across tasks.
  arXiv  Detail & Related papers  (2026-08-04T14:36:34Z)
• From Textural Counterpoint to Feature Encoding: A Multi-Dimensional Machine Representation Study of Haydn's "The Lark" Integrating Electroacoustic Analysis [11.369230577047423]
  This paper conducts an interdisciplinary analysis of Haydn's String Quartet in D Major, The Lark.<n>The study first utilizes auditory analysis to dissect the counterpoint morphology of the leading voice.<n>The paper introduces a fundamentally new approach to low-level computer feature extraction.
  arXiv  Detail & Related papers  (2026-07-07T06:58:46Z)
• Audio-Visual Intelligence in Large Foundation Models [123.78167820053545]
  Audio-Visual Intelligence (AVI) has emerged as a central frontier in artificial intelligence.<n>Despite rapid progress, the literature remains fragmented, spanning diverse tasks, inconsistent, and heterogeneous evaluation practices.<n>This survey provides the first comprehensive review of AVI through the lens of large foundation models.
  arXiv  Detail & Related papers  (2026-05-05T17:59:16Z)
• UniSonate: A Unified Model for Speech, Music, and Sound Effect Generation with Text Instructions [55.622295453533475]
  We introduce UniSonate, a unified flow-matching framework capable of synthesizing speech, music, and sound effects.<n>We propose a novel dynamic token injection mechanism that projects unstructured environmental sounds into a structured temporal latent space.<n>Experiments demonstrate that UniSonate achieves state-of-the-art performance in instruction-based TTS and TTM.
  arXiv  Detail & Related papers  (2026-04-24T04:26:04Z)
• Exploring Definitions of Quality and Diversity in Sonic Measurement Spaces [1.7411855207380258]
  Digital sound synthesis presents the opportunity to explore vast parameter spaces containing millions of configurations. Quality diversity (QD) evolutionary algorithms offer a promising approach to harness this potential, yet their success hinges on appropriate sonic feature representations.<n>This work investigates unsupervised dimensionality reduction methods for automatically defining and dynamically reconfiguring sonic behaviour spaces during QD search.<n>Results contribute to automated sonic discovery systems capable of exploring vast parameter spaces without manual intervention or supervised training constraints.
  arXiv  Detail & Related papers  (2025-12-02T13:57:08Z)
• High-Quality Sound Separation Across Diverse Categories via Visually-Guided Generative Modeling [65.02357548201188]
  We propose DAVIS, a Diffusion-based Audio-VIsual Separation framework that solves the audio-visual sound source separation task through generative learning.<n>Our framework operates by synthesizing the desired separated sound spectrograms directly from a noise distribution, conditioned concurrently on the mixed audio input and associated visual information.
  arXiv  Detail & Related papers  (2025-09-26T08:46:00Z)
• Discrete Audio Tokens: More Than a Survey! [137.3721175670642]
  This paper presents a systematic review and benchmark of discrete audio tokenizers.<n>It covers speech, music, and general audio domains.<n>We propose a taxonomy of tokenization approaches based on encoder-decoder, quantization techniques, training paradigm, streamability, and application domains.
  arXiv  Detail & Related papers  (2025-06-12T01:35:43Z)
• LVNS-RAVE: Diversified audio generation with RAVE and Latent Vector Novelty Search [0.5624791703748108]
  We propose LVNS-RAVE, a method to combine Evolutionary Algorithms and Generative Deep Learning to produce realistic sounds. The proposed algorithm can be a creative tool for sound artists and musicians.
  arXiv  Detail & Related papers  (2024-04-22T10:20:41Z)
• Sound Model Factory: An Integrated System Architecture for Generative Audio Modelling [4.193940401637568]
  We introduce a new system for data-driven audio sound model design built around two different neural network architectures. The objective of the system is to generate interactively controllable sound models given (a) a range of sounds the model should be able to synthesize, and (b) a specification of the parametric controls for navigating that space of sounds.
  arXiv  Detail & Related papers  (2022-06-27T07:10:22Z)
• Learning to Listen: Modeling Non-Deterministic Dyadic Facial Motion [89.01668641930206]
  We present a framework for modeling interactional communication in dyadic conversations. We autoregressively output multiple possibilities of corresponding listener motion. Our method organically captures the multimodal and non-deterministic nature of nonverbal dyadic interactions.
  arXiv  Detail & Related papers  (2022-04-18T17:58:04Z)
• Audio-Visual Fusion Layers for Event Type Aware Video Recognition [86.22811405685681]
  We propose a new model to address the multisensory integration problem with individual event-specific layers in a multi-task learning scheme. We show that our network is formulated with single labels, but it can output additional true multi-labels to represent the given videos.
  arXiv  Detail & Related papers  (2022-02-12T02:56:22Z)
• EEGminer: Discovering Interpretable Features of Brain Activity with Learnable Filters [72.19032452642728]
  We propose a novel differentiable EEG decoding pipeline consisting of learnable filters and a pre-determined feature extraction module. We demonstrate the utility of our model towards emotion recognition from EEG signals on the SEED dataset and on a new EEG dataset of unprecedented size. The discovered features align with previous neuroscience studies and offer new insights, such as marked differences in the functional connectivity profile between left and right temporal areas during music listening.
  arXiv  Detail & Related papers  (2021-10-19T14:22:04Z)
• Timbre latent space: exploration and creative aspects [1.3764085113103222]
  Recent studies show the ability of unsupervised models to learn invertible audio representations using Auto-Encoders. New possibilities for timbre manipulations are enabled with generative neural networks.
  arXiv  Detail & Related papers  (2020-08-04T07:08:04Z)
• Modeling Musical Structure with Artificial Neural Networks [0.0]
  I explore the application of artificial neural networks to different aspects of musical structure modeling. I show how a connectionist model, the Gated Autoencoder (GAE), can be employed to learn transformations between musical fragments. I propose a special predictive training of the GAE, which yields a representation of polyphonic music as a sequence of intervals.
  arXiv  Detail & Related papers  (2020-01-06T18:35:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.