What Makes a Good Layer? Assessing the Layer-Wise Intrinsic Properties of Music Foundation Models

• URL: http://arxiv.org/abs/2608.14819v1
• Date: Fri, 14 Aug 2026 18:46:22 GMT
• Title: What Makes a Good Layer? Assessing the Layer-Wise Intrinsic Properties of Music Foundation Models
• Abstract summary: Music foundation models are commonly used as frozen audio feature extractors, yet selecting which layer to extract from remains largely intrinsic.<n>We conduct a systematic analysis of 12 music foundation models spanning three pre-training paradigms.<n>We show that intrinsic metrics can serve as effective proxies for layer selection, matching or outperforming trainable multi-layer fusion methods.
• Score: 11.33270501801585
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Music foundation models are commonly used as frozen audio feature extractors, yet selecting which layer to extract from remains largely heuristic. Current practice defaults to fixed depths or multi-layer fusion, with limited understanding of why certain layers transfer better across downstream tasks or how representation quality varies with depth and pre-training paradigm. We conduct a systematic layer-wise analysis of 12 music foundation models spanning three pre-training paradigms (masked modeling, autoregressive modeling, and contrastive learning), characterizing their hidden representations through intrinsic geometric and transformation-based properties. Correlating label-free representation-quality metrics with layer-wise performance across 15 downstream tasks, we find that several metrics track layer quality for genre classification, emotion recognition, automatic tagging, and beat tracking, albeit with varying strength across tasks and pre-training paradigms. However, all metrics fail on tonal tasks such as key estimation and chord recognition, indicating that no single property serves as a general proxy for representation quality across music information retrieval tasks. To address this gap, we introduce a pitch-transposition equivariance measure that captures properties missed by these standard metrics, providing a consistent indicator of tonal quality across model families. Finally, we show that intrinsic metrics can serve as effective proxies for layer selection, matching or outperforming trainable multi-layer fusion methods, particularly in limited-data settings.

Related papers

• Do All Individual Layers Help? An Empirical Study of Task-Interfering Layers in Vision-Language Models [51.754991950934375]
  In a pretrained VLM, all layers are engaged by default to make predictions on downstream tasks.<n>We find that intervening on a single layer, such as by zeroing its parameters, can improve the performance on certain tasks.<n>We propose TaLo, a training-free, test-time adaptation method that dynamically identifies and bypasses the most interfering layer for a given task.
  arXiv  Detail & Related papers  (2026-02-01T11:37:05Z)
• Layer by Layer: Uncovering Hidden Representations in Language Models [28.304269706993942]
  We show that intermediate layers can encode even richer representations, often improving performance on a range of downstream tasks.<n>Our framework highlights how each layer balances information compression and signal preservation, revealing why mid-depth embeddings can exceed the last layer's performance.
  arXiv  Detail & Related papers  (2025-02-04T05:03:42Z)
• Persistent Topological Features in Large Language Models [0.6597195879147556]
  We introduce topological descriptors that measure how topological features, $p$-dimensional holes, persist and evolve throughout the layers.<n>This offers a statistical perspective on how prompts are rearranged and their relative positions changed in the representation space.<n>As a showcase application, we use zigzag persistence to establish a criterion for layer pruning, achieving results comparable to state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-10-14T19:46:23Z)
• MTP: Advancing Remote Sensing Foundation Model via Multi-Task Pretraining [73.81862342673894]
  Foundation models have reshaped the landscape of Remote Sensing (RS) by enhancing various image interpretation tasks. transferring the pretrained models to downstream tasks may encounter task discrepancy due to their formulation of pretraining as image classification or object discrimination tasks. We conduct multi-task supervised pretraining on the SAMRS dataset, encompassing semantic segmentation, instance segmentation, and rotated object detection. Our models are finetuned on various RS downstream tasks, such as scene classification, horizontal and rotated object detection, semantic segmentation, and change detection.
  arXiv  Detail & Related papers  (2024-03-20T09:17:22Z)
• Stochastic Layers in Vision Transformers [85.38733795180497]
  We introduce fully layers in vision transformers, without causing any severe drop in performance. The additionality boosts the robustness of visual features and strengthens privacy. We use our features for three different applications, namely, adversarial robustness, network calibration, and feature privacy.
  arXiv  Detail & Related papers  (2021-12-30T16:07:59Z)
• Hierarchical Variational Memory for Few-shot Learning Across Domains [120.87679627651153]
  We introduce a hierarchical prototype model, where each level of the prototype fetches corresponding information from the hierarchical memory. The model is endowed with the ability to flexibly rely on features at different semantic levels if the domain shift circumstances so demand. We conduct thorough ablation studies to demonstrate the effectiveness of each component in our model.
  arXiv  Detail & Related papers  (2021-12-15T15:01:29Z)
• Mitigating Generation Shifts for Generalized Zero-Shot Learning [52.98182124310114]
  Generalized Zero-Shot Learning (GZSL) is the task of leveraging semantic information (e.g., attributes) to recognize the seen and unseen samples, where unseen classes are not observable during training. We propose a novel Generation Shifts Mitigating Flow framework for learning unseen data synthesis efficiently and effectively. Experimental results demonstrate that GSMFlow achieves state-of-the-art recognition performance in both conventional and generalized zero-shot settings.
  arXiv  Detail & Related papers  (2021-07-07T11:43:59Z)
• CharacterGAN: Few-Shot Keypoint Character Animation and Reposing [64.19520387536741]
  We introduce CharacterGAN, a generative model that can be trained on only a few samples of a given character. Our model generates novel poses based on keypoint locations, which can be modified in real time while providing interactive feedback. We show that our approach outperforms recent baselines and creates realistic animations for diverse characters.
  arXiv  Detail & Related papers  (2021-02-05T12:38:15Z)
• Music FaderNets: Controllable Music Generation Based On High-Level Features via Low-Level Feature Modelling [5.88864611435337]
  We present a framework that can learn high-level feature representations with a limited amount of data. We refer to our proposed framework as Music FaderNets, which is inspired by the fact that low-level attributes can be continuously manipulated. We demonstrate that the model successfully learns the intrinsic relationship between arousal and its corresponding low-level attributes.
  arXiv  Detail & Related papers  (2020-07-29T16:01:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.