Structural Bottlenecks on Frequency Representation in End-to-End Audio Models

• URL: http://arxiv.org/abs/2607.08545v1
• Date: Thu, 09 Jul 2026 14:38:16 GMT
• Title: Structural Bottlenecks on Frequency Representation in End-to-End Audio Models
• Abstract summary: We show that state-of-the-art convolutional encoders impose two structural bottlenecks on access to frequency-localized primitives.<n>We introduce Gabor Latent Refactorization (GLRF), a lightweight intervention that re-expresses encoder latents in a frequency-localized basis.<n>These results show that the encoders in question predictably degrade access to frequency-localized primitives, entangling the features that depend on them.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: End-to-end neural audio models achieve high-fidelity compression and generation. We might read that performance as evidence they directly represent interpretable features such as pitch and timbre, but a model can produce plausible outputs without doing so. A model may encode these features in any reachable basis, but regardless of which, the features are well described as compositions of time-frequency-localized primitives. Whether state-of-the-art encoders preserve access to these primitives, and thus to compositions of them, remains unclear. Through theoretical analysis and controlled experiments, we show that several state-of-the-art strided convolutional encoders impose two structural bottlenecks, both predictable from architecture and signal structure, on access to these primitives: (1) they collapse primitives into alias equivalence classes, establishing a bound on representational capacity, and (2) they limit the frequency resolution available to learned filters, restricting separability. For well structured data, we find collapse rates of 31-35% and filter bandwidths 10-35x above the theoretical resolution bound, confirming that both bottlenecks arise under realistic signal conditions. We then introduce Gabor Latent Refactorization (GLRF), a lightweight post-hoc intervention that re-expresses encoder latents in a frequency-localized basis, reducing filter bandwidths from 10-35x to 1.5-3x of the theoretical resolution bound while preserving reconstruction fidelity and improving control over attributes like pitch. These results show that the encoders in question predictably degrade access to frequency-localized primitives, entangling the features that depend on them, and that a lightweight, retraining-free intervention can recover much of that access, improving steerability and interpretability.

Related papers

• Survey of Novel Deep Learning Architectures for Denoising Gravitational-wave Signals [0.0]
  We present the first controlled comparison of five neural-network architectures for gravitational-wave denoising.<n>We train identically across the full astrophysically-motivated spinning binary-black-hole parameter space.<n>A unifying principle emerges: matching network structure to the spectral anatomy of a coalescence outperforms brute-force model scaling.
  arXiv  Detail & Related papers  (2026-09-07T18:00:10Z)
• ScalingAttention: Discovering Intrinsic Sparse Attention Topology for Video Diffusion Transformers [7.879917783645986]
  Diffusion Transformers (DiTs) have revolutionized high-fidelity video generation, their reliance on 3D full attention creates a quadratic computational bottleneck.<n>Existing sparse methods face a dilemma: dynamic pruning suffers from prohibitive runtime overhead and memory fragmentation, while statics fail to capture fine-grained dependencies.<n>In this work, we propose ScalingAttention, a training-free framework grounded in a key inductive bias.<n> Experiments on Wan2.1 show up to 1.90X end-to-end speedup with superior fidelity, establishing a new frontier over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-06-22T08:32:07Z)
• Parameter-efficient Dual-encoder Architecture with Differentiable Choquet Integral Fusion for Underwater Acoustic Classification [43.92337343771302]
  This paper proposes a dual-encoder neural architecture to simultaneously process acoustic waveforms and spectrograms.<n>By dynamically shifting attention to the representation least corrupted by potential asymmetric channel distortions, the proposed gating mechanism mitigates the non-stationary challenges of the underwater environment.
  arXiv  Detail & Related papers  (2026-06-01T14:49:38Z)
• Physics from Video: Identifiability of Time-Invariant Second-Order ODEs under Minimal Trajectory Conditions [60.2784641643737]
  Bridging the gap between visual realism and physical understanding is a core challenge for video-based world models.<n>We study the structural identifiability of continuous-time physical laws from raw pixels.<n>We prove that a level-set slope-coverage condition ensures the learned latent space is locally affine to the true physical state.
  arXiv  Detail & Related papers  (2026-05-27T13:22:29Z)
• DINO-SAE: DINO Spherical Autoencoder for High-Fidelity Image Reconstruction and Generation [47.409626500688866]
  We present the DINO Spherical Autoencoder (DINO-SAE), a framework that bridges semantic representation and pixel-level reconstruction.<n>Our approach achieves state-of-the-art reconstruction quality, reaching 0.37 rFID and 26.2 dB PSNR, while maintaining strong semantic alignment to the pretrained VFM.
  arXiv  Detail & Related papers  (2026-01-30T12:25:34Z)
• FLaTEC: Frequency-Disentangled Latent Triplanes for Efficient Compression of LiDAR Point Clouds [52.997038111673966]
  FLaTEC is a frequency-aware compression model that enables the compression of a full scan with high compression ratios.<n>We convert voxelized embeddings into triplane representations to reduce sparsity, computational cost, and storage requirements.<n>Our method achieves state-of-the-art rate-distortion performance and outperforms the standard codecs by 78% and 94% in BD-rate on both datasets.
  arXiv  Detail & Related papers  (2025-11-25T08:37:49Z)
• ECHO: Frequency-aware Hierarchical Encoding for Variable-length Signals [8.411477071838592]
  We propose a novel foundation model ECHO that integrates an advanced band-split architecture with frequency positional embeddings.<n>We evaluate our method on various kinds of machine signal datasets.
  arXiv  Detail & Related papers  (2025-08-20T13:10:44Z)
• Re-Bottleneck: Latent Re-Structuring for Neural Audio Autoencoders [14.777092647088756]
  We propose a simple, post-hoc framework to modify the bottleneck of a pre-trained autoencoder.<n>Our method introduces a "Re-Bottleneck", an inner bottleneck trained exclusively through latent space losses to instill user-defined structure.<n>Ultimately, our Re-Bottleneck framework offers a flexible and efficient way to tailor representations of neural audio models.
  arXiv  Detail & Related papers  (2025-07-10T15:47:43Z)
• Fundamental Limits of Two-layer Autoencoders, and Achieving Them with Gradient Methods [91.54785981649228]
  This paper focuses on non-linear two-layer autoencoders trained in the challenging proportional regime. Our results characterize the minimizers of the population risk, and show that such minimizers are achieved by gradient methods. For the special case of a sign activation function, our analysis establishes the fundamental limits for the lossy compression of Gaussian sources via (shallow) autoencoders.
  arXiv  Detail & Related papers  (2022-12-27T12:37:34Z)
• High Fidelity Neural Audio Compression [92.4812002532009]
  We introduce a state-of-the-art real-time, high-fidelity, audio leveraging neural networks. It consists in a streaming encoder-decoder architecture with quantized latent space trained in an end-to-end fashion. We simplify and speed-up the training by using a single multiscale spectrogram adversary.
  arXiv  Detail & Related papers  (2022-10-24T17:52:02Z)
• Conditioning Trick for Training Stable GANs [70.15099665710336]
  We propose a conditioning trick, called difference departure from normality, applied on the generator network in response to instability issues during GAN training. We force the generator to get closer to the departure from normality function of real samples computed in the spectral domain of Schur decomposition.
  arXiv  Detail & Related papers  (2020-10-12T16:50:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.