Melody or Machine: Detecting Synthetic Music with Dual-Stream Contrastive Learning

• URL: http://arxiv.org/abs/2512.00621v1
• Date: Sat, 29 Nov 2025 20:25:20 GMT
• Title: Melody or Machine: Detecting Synthetic Music with Dual-Stream Contrastive Learning
• Abstract summary: Melody or Machine is a new large-scale benchmark of over 130,000 songs.<n> CLAM is a novel dual-stream detection architecture.<n>It achieves an F1 score of 0.925 on our challenging MoM benchmark.
• Score: 0.4310167974376404
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The rapid evolution of end-to-end AI music generation poses an escalating threat to artistic authenticity and copyright, demanding detection methods that can keep pace. While foundational, existing models like SpecTTTra falter when faced with the diverse and rapidly advancing ecosystem of new generators, exhibiting significant performance drops on out-of-distribution (OOD) content. This generalization failure highlights a critical gap: the need for more challenging benchmarks and more robust detection architectures. To address this, we first introduce Melody or Machine (MoM), a new large-scale benchmark of over 130,000 songs (6,665 hours). MoM is the most diverse dataset to date, built with a mix of open and closed-source models and a curated OOD test set designed specifically to foster the development of truly generalizable detectors. Alongside this benchmark, we introduce CLAM, a novel dual-stream detection architecture. We hypothesize that subtle, machine-induced inconsistencies between vocal and instrumental elements, often imperceptible in a mixed signal, offer a powerful tell-tale sign of synthesis. CLAM is designed to test this hypothesis by employing two distinct pre-trained audio encoders (MERT and Wave2Vec2) to create parallel representations of the audio. These representations are fused by a learnable cross-aggregation module that models their inter-dependencies. The model is trained with a dual-loss objective: a standard binary cross-entropy loss for classification, complemented by a contrastive triplet loss which trains the model to distinguish between coherent and artificially mismatched stream pairings, enhancing its sensitivity to synthetic artifacts without presuming a simple feature alignment. CLAM establishes a new state-of-the-art in synthetic music forensics. It achieves an F1 score of 0.925 on our challenging MoM benchmark.

Related papers

• Diff-V2M: A Hierarchical Conditional Diffusion Model with Explicit Rhythmic Modeling for Video-to-Music Generation [26.273309051211204]
  Video-to-music (V2M) generation aims to create music that aligns with visual content.<n>We propose Diff-V2M, a general V2M framework based on a hierarchical conditional diffusion model.<n>For rhythm modeling, we begin by evaluating several rhythmic representations, including low-resolution mel-spectrograms, tempograms, and onset detection functions (ODF)
  arXiv  Detail & Related papers  (2025-11-12T08:02:06Z)
• Beyond Frequency: Scoring-Driven Debiasing for Object Detection via Blueprint-Prompted Image Synthesis [97.37770785712475]
  We present a generation-based debiasing framework for object detection.<n>Our method significantly narrows the performance gap for underrepresented object groups.
  arXiv  Detail & Related papers  (2025-10-21T02:19:12Z)
• Every Step Counts: Decoding Trajectories as Authorship Fingerprints of dLLMs [63.82840470917859]
  We show that the decoding mechanism of dLLMs can be used as a powerful tool for model attribution.<n>We propose a novel information extraction scheme called the Directed Decoding Map (DDM), which captures structural relationships between decoding steps and better reveals model-specific behaviors.
  arXiv  Detail & Related papers  (2025-10-02T06:25:10Z)
• ECHO: Frequency-aware Hierarchical Encoding for Variable-length Signals [8.411477071838592]
  We propose a novel foundation model ECHO that integrates an advanced band-split architecture with frequency positional embeddings.<n>We evaluate our method on various kinds of machine signal datasets.
  arXiv  Detail & Related papers  (2025-08-20T13:10:44Z)
• Scaling Self-Supervised Representation Learning for Symbolic Piano Performance [52.661197827466886]
  We study the capabilities of generative autoregressive transformer models trained on large amounts of symbolic solo-piano transcriptions.<n>We use a comparatively smaller, high-quality subset to finetune models to produce musical continuations, perform symbolic classification tasks, and produce general-purpose contrastive MIDI embeddings.
  arXiv  Detail & Related papers  (2025-06-30T14:00:14Z)
• Seeing What Matters: Generalizable AI-generated Video Detection with Forensic-Oriented Augmentation [31.737159092430108]
  We study different generative architectures, searching and identifying discriminative features that are unbiased, robust to impairments, and shared across models.<n>We introduce a novel data augmentation strategy based on the wavelet decomposition and replace specific frequency-related bands to drive the model to exploit more relevant forensic cues.<n>Our method achieves a significant accuracy improvement over state-of-the-art detectors and obtains excellent results even on very recent generative models.
  arXiv  Detail & Related papers  (2025-06-20T07:36:59Z)
• Aligning Text-to-Music Evaluation with Human Preferences [63.08368388389259]
  We study the design space of reference-based divergence metrics for evaluating generative acoustic text-to-music (TTM) models.<n>We find that not only is the standard FAD setup inconsistent on both synthetic and human preference data, but that nearly all existing metrics fail to effectively capture desiderata.<n>We propose a new metric, the MAUVE Audio Divergence (MAD), computed on representations from a self-supervised audio embedding model.
  arXiv  Detail & Related papers  (2025-03-20T19:31:04Z)
• HFMF: Hierarchical Fusion Meets Multi-Stream Models for Deepfake Detection [4.908389661988192]
  HFMF is a comprehensive two-stage deepfake detection framework.<n>It integrates vision Transformers and convolutional nets through a hierarchical feature fusion mechanism.<n>We demonstrate that our architecture achieves superior performance across diverse dataset benchmarks.
  arXiv  Detail & Related papers  (2025-01-10T00:20:29Z)
• SpecDiff-GAN: A Spectrally-Shaped Noise Diffusion GAN for Speech and Music Synthesis [0.0]
  We introduce SpecDiff-GAN, a neural vocoder based on HiFi-GAN. We show the merits of our proposed model for speech and music synthesis on several datasets.
  arXiv  Detail & Related papers  (2024-01-30T09:17:57Z)
• Self-Distilled Masked Auto-Encoders are Efficient Video Anomaly Detectors [117.61449210940955]
  We propose an efficient abnormal event detection model based on a lightweight masked auto-encoder (AE) applied at the video frame level. We introduce an approach to weight tokens based on motion gradients, thus shifting the focus from the static background scene to the foreground objects. We generate synthetic abnormal events to augment the training videos, and task the masked AE model to jointly reconstruct the original frames.
  arXiv  Detail & Related papers  (2023-06-21T06:18:05Z)
• Multi-instrument Music Synthesis with Spectrogram Diffusion [19.81982315173444]
  We focus on a middle ground of neural synthesizers that can generate audio from MIDI sequences with arbitrary combinations of instruments in realtime. We use a simple two-stage process: MIDI to spectrograms with an encoder-decoder Transformer, then spectrograms to audio with a generative adversarial network (GAN) spectrogram inverter. We find this to be a promising first step towards interactive and expressive neural synthesis for arbitrary combinations of instruments and notes.
  arXiv  Detail & Related papers  (2022-06-11T03:26:15Z)
• Attention Bottlenecks for Multimodal Fusion [90.75885715478054]
  Machine perception models are typically modality-specific and optimised for unimodal benchmarks. We introduce a novel transformer based architecture that uses fusion' for modality fusion at multiple layers. We conduct thorough ablation studies, and achieve state-of-the-art results on multiple audio-visual classification benchmarks.
  arXiv  Detail & Related papers  (2021-06-30T22:44:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.