From General-Purpose Audio Tagging to Spatially Grounded Sound Event Localization and Detection

• URL: http://arxiv.org/abs/2606.27751v1
• Date: Fri, 26 Jun 2026 06:12:43 GMT
• Title: From General-Purpose Audio Tagging to Spatially Grounded Sound Event Localization and Detection
• Abstract summary: The proposed AT2SELD framework couples a pretrained AT backbone with compact First-Order Ambisonics (FOA) spatial processing.<n>It characterizes how semantic audio priors support localization-aware scene analysis under data, computation, and deployment constraints.
• Score: 6.515657411366296
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: This report investigates the extension of pretrained General-Purpose Audio Tagging (GP-AT) models toward spatially grounded Sound Event Localization and Detection (SELD). The proposed AT2SELD framework couples a pretrained AT backbone with compact First-Order Ambisonics (FOA) spatial processing, track-wise SED and Cartesian DOA estimation, permutation aware supervision, and calibration. It characterizes how semantic audio priors support localization-aware scene analysis under data, computation, and deployment constraints. The framework is developed through informed multi-stage Neural Architecture Search (NAS). Stage 1 shows that spectral FOA descriptors, based on magnitude, phase, and Intensity Vectors (IVs), provide the most reliable interface for semantic-to-spatial transfer. Stage 2 identifies early residual spatial encoding as the main capacity-sensitive component, while late track-wise abstraction and recurrent smoothing act mainly as refinement stages. Stage 3 shows that late cross-stitch coupling improves semantic-spatial interaction, whereas early fusion is costlier and less effective. Diagnostic evaluation analyzes the selected architecture under class balancing, focal loss, activity-conditioned DOA supervision, threshold calibration, and transfer across STARSS23, TAU2019, TAU-NIGENS2020, and TAU-NIGENS2021. Focal loss improves the activity point, active-only DOA supervision mitigates inactive target dominance, and validation-selected thresholds recover calibration without replacing spatial learning. Cross-dataset and oracle-activity analyses indicate strong fixed source localization on TAU2019, transferable representations from TAU NIGENS2021, and meaningful but uncertain behavior on STARSS23. Overall, GP-AT priors appear promising for SELD design when embedded in spatial-aware architectures and optimized through integrated calibration and deployment oriented strategies.

Related papers

• SSC-Priors: Exploring Semantic and Visibility Priors to Boost Lidar Semantic Scene Completion [61.665449818739155]
  This paper investigates easy strategies to boost the performance of existing networks for lidar semantic scene completion (SSC)<n>We use semantic pseudo-labels and sensor visibility information to improve existing approaches.<n>We show that endowing input point clouds with semantic pseudo-labels from off-the-shelf segmenters significantly improves the performance of existing SSC models.
  arXiv  Detail & Related papers  (2026-09-15T16:37:46Z)
• FLORO: A Multimodal Geospatial Foundation Model for Ecological Remote Sensing Across Sensors and Scales [65.4821703903285]
  FLORO is a multimodal foundation model designed to learn transferable representations from a small but highly diverse remote sensing corpus.<n> FLORO is pretrained using masked autoencoding on a heterogeneous combination of Sentinel-1, Sentinel-2, SkySAT imagery, elevation, and UAV-derived data.<n>We evaluated FLORO on the PANGAEA benchmark under a frozen-encoder protocol across scene classification, segmentation, and regression tasks.
  arXiv  Detail & Related papers  (2026-05-27T08:55:54Z)
• RELO: Reinforcement Learning to Localize for Visual Object Tracking [70.27265738642956]
  We introduce RELO, a REinforcement-learning-to-LOcalize method for visual object tracking.<n>RELO replaces handcrafted spatial priors with a localization policy learned over spatial positions.<n>We show that RELO achieves 57.5% AUC on LaSOText without template updates.
  arXiv  Detail & Related papers  (2026-05-08T07:34:29Z)
• AHPA: Adaptive Hierarchical Prior Alignment for Diffusion Transformers [24.937985157569823]
  We argue that such timestep-agnostic alignment is suboptimal because the useful granularity of representation supervision changes systematically with the signal-to-noise ratio.<n>In high-noise regimes, diffusion models benefit more from coarse semantic and layout-level anchoring, whereas in low-noise regimes, the training signal should emphasize spatially detailed and structurally faithful refinement.<n>This non-stationary alignment behavior creates a representational mismatch for static single-level supervisors.
  arXiv  Detail & Related papers  (2026-05-05T03:07:29Z)
• VFM$^{4}$SDG: Unveiling the Power of VFMs for Single-Domain Generalized Object Detection [33.39250067795076]
  In real-world scenarios, continual changes in weather, illumination, and imaging conditions cause significant domain shifts.<n>We propose a dual-prior learning framework for generalized object detection (SDGOD)<n>In the encoding stage, we propose Cross-domain Stable Prior Distillation to enhance the robustness of object-background and inter-instance modeling.<n>In the decoding stage, we propose Semantic-Contextual Prior-based Query Enhancement.
  arXiv  Detail & Related papers  (2026-04-23T10:04:36Z)
• Foundation Model Priors Enhance Object Focus in Feature Space for Source-Free Object Detection [38.14795337940857]
  Domain shift reduces the detector's ability to maintain strong object-focused representations.<n>FALCON-SFOD is a framework designed to enhance object-focused adaptation under domain shift.
  arXiv  Detail & Related papers  (2025-12-19T12:30:29Z)
• FoBa: A Foreground-Background co-Guided Method and New Benchmark for Remote Sensing Semantic Change Detection [48.06921153684768]
  We present a new benchmark for remote sensing semantic change detection (SCD) called LevirSCD.<n>The dataset covers 16 change categories and 210 specific change types, with more fine-grained class definitions.<n>We propose a foreground-background co-guided SCD (FoBa) method, which leverages foregrounds enriched with contextual information to guide the model.<n>FoBa achieves competitive results compared to current SOTA methods, with improvements of 1.48%, 3.61%, and 2.81% in the SeK metric, respectively.
  arXiv  Detail & Related papers  (2025-09-19T09:19:57Z)
• An ocean front detection and tracking algorithm [4.0604303269549185]
  This paper proposes the Bayesian Front Detection and Tracking framework with Metric Space Analysis.<n>BFDTMSA reduces over-detection by $73%$ compared to histogram-based methods.<n>The open-source release bridges a critical gap in reproducible oceanographic research.
  arXiv  Detail & Related papers  (2025-02-21T07:00:09Z)
• Spatial-Aware Token for Weakly Supervised Object Localization [137.0570026552845]
  We propose a task-specific spatial-aware token to condition localization in a weakly supervised manner. Experiments show that the proposed SAT achieves state-of-the-art performance on both CUB-200 and ImageNet, with 98.45% and 73.13% GT-known Loc.
  arXiv  Detail & Related papers  (2023-03-18T15:38:17Z)
• Relation Matters: Foreground-aware Graph-based Relational Reasoning for Domain Adaptive Object Detection [81.07378219410182]
  We propose a new and general framework for DomainD, named Foreground-aware Graph-based Reasoning (FGRR) FGRR incorporates graph structures into the detection pipeline to explicitly model the intra- and inter-domain foreground object relations. Empirical results demonstrate that the proposed FGRR exceeds the state-of-the-art on four DomainD benchmarks.
  arXiv  Detail & Related papers  (2022-06-06T05:12:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.