Weakly Guided and Autoregressive Beamformer Parameterization for Generalizable Moving Speaker Extraction in Higher-Order Ambisonics

• URL: http://arxiv.org/abs/2607.04471v1
• Date: Sun, 05 Jul 2026 19:32:57 GMT
• Title: Weakly Guided and Autoregressive Beamformer Parameterization for Generalizable Moving Speaker Extraction in Higher-Order Ambisonics
• Abstract summary: Linear spatial filters (beamformers) enable robust, generalizable and interpretable speech enhancement.<n>Modern beamformers are often parameterized by deep neural networks, whose performance degrades in dynamic scenarios.<n>We propose a data-driven beamforming pipeline, which only requires an estimate of the target's initial direction.
• Score: 26.916336572507692
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Linear spatial filters (beamformers) enable robust, generalizable and interpretable speech enhancement with performance guarantees under ideal parameterization. Modern beamformers are often parameterized by deep neural networks, whose performance degrades in dynamic scenarios with multiple moving speakers of unknown directions. We propose a data-driven beamforming pipeline, which only requires an estimate of the target's initial direction. Building on a higher-order ambisonics representation, we show that neural temporal-spectral processing can be decoupled from linear spatial processing, and thereby achieve generalizable and array-agnostic enhancement. By incorporating autoregression into a frame-wise causal framework, we maintain consistent performance throughout fast speaker motion and long recordings. Evaluation on synthetic data demonstrates robust enhancement under challenging conditions with closely spaced and crossing speakers. Real-world recordings in a dynamic office meeting scenario complement these findings and show generalizability across varying ambisonics orders.

Related papers

• Dynamic-Robust Photometric-Semantic Reconstruction for Open-Vocabulary 3D Scene Understanding [63.599342750566485]
  SPAR is a novel joint semantic-geometric encoding architecture that explicitly isolates transient dynamic noise prior to latent space aggregation.<n>Our end-to-end approach achieves exceptional novel view synthesis quality, yielding a PSNR of 22.15 dB and 23.33 dB given only 3 and 4 input views respectively.
  arXiv  Detail & Related papers  (2026-08-29T09:58:24Z)
• DAVE: A Decoupled Audio-Visual Enhancement Framework for Real-World Speech Separation [4.041253468990907]
  We present DAVE, a decoupled audio-visual enhancement framework for real-world speech separation.<n>To address the data scarcity issue, we construct DAVE-Corpus, a large-scale training corpus with 219,411 mixtures generated from public meeting corpora.<n>We further develop a certified selective enhancement chain that applies scene routing, GAN-based denoising, and loudness normalization only within the no-reference partition.
  arXiv  Detail & Related papers  (2026-08-10T08:41:28Z)
• Enhancing Flow Matching with A Unified Guidance Framework for Efficient and Robust Speech Synthesis [11.116469047991798]
  Flow Matching (FM) has emerged as a powerful paradigm for speech generation but remains constrained by high inference and latency.<n>We propose a unified guidance framework that enhances generation efficiency and robustness through two complementary strategies.
  arXiv  Detail & Related papers  (2026-07-01T03:02:31Z)
• Autoregressive Guidance of Deep Spatially Selective Filters using Bayesian Tracking for Efficient Extraction of Moving Speakers [28.32055656460997]
  In this work, we investigate strategies to incorporate the enhanced signal into lightweight tracking algorithms and autoregressively guide deep spatial filters.<n>Results validate that the autoregressive incorporation significantly improves the accuracy of our Bayesian trackers, resulting in superior enhancement with none or only negligibly increased computational overhead.
  arXiv  Detail & Related papers  (2026-03-24T21:19:45Z)
• Cross-Scenario Deraining Adaptation with Unpaired Data: Superpixel Structural Priors and Multi-Stage Pseudo-Rain Synthesis [48.809049043865905]
  Image deraining plays a pivotal role in low-level computer vision.<n>Deep learning paradigms often suffer from severe performance degradation when generalized to unseen Out-of-Distribution scenarios.<n>This paper proposes a pioneering cross-scenario deraining adaptation framework.
  arXiv  Detail & Related papers  (2026-03-23T07:38:55Z)
• Adaptive Rotary Steering with Joint Autoregression for Robust Extraction of Closely Moving Speakers in Dynamic Scenarios [28.32055656460997]
  Latest advances in deep spatial filtering for Ambisonics demonstrate strong performance in stationary multi-speaker scenarios.<n>We propose to automate this rotary steering using an interleaved tracking algorithm conditioned on the target's initial direction.<n>For nearby or crossing speakers, robust tracking becomes difficult and spatial cues less effective for enhancement.
  arXiv  Detail & Related papers  (2026-01-18T10:38:49Z)
• Towards Arbitrary Motion Completing via Hierarchical Continuous Representation [64.6525112550758]
  We propose a novel parametric activation-induced hierarchical implicit representation framework, called NAME, based on Implicit Representations (INRs)<n>Our method introduces a hierarchical temporal encoding mechanism that extracts features from motion sequences at multiple temporal scales, enabling effective capture of intricate temporal patterns.
  arXiv  Detail & Related papers  (2025-12-24T14:07:04Z)
• Efficient Speech Language Modeling via Energy Distance in Continuous Latent Space [74.12387631212609]
  We introduce SLED, an alternative approach to speech language modeling by encoding speech waveforms into sequences of continuous latent representations.<n>SLED avoids discretization errors and eliminates the need for the complicated hierarchical architectures common in existing speech language models.<n> Empirical results demonstrate that SLED achieves strong performance in both zero-shot and streaming speech synthesis.
  arXiv  Detail & Related papers  (2025-05-19T14:38:59Z)
• Dita: Scaling Diffusion Transformer for Generalist Vision-Language-Action Policy [73.75271615101754]
  We present Dita, a scalable framework that leverages Transformer architectures to directly denoise continuous action sequences.<n>Dita employs in-context conditioning -- enabling fine-grained alignment between denoised actions and raw visual tokens from historical observations.<n>Dita effectively integrates cross-embodiment datasets across diverse camera perspectives, observation scenes, tasks, and action spaces.
  arXiv  Detail & Related papers  (2025-03-25T15:19:56Z)
• Diffusion Transformer Policy [48.50988753948537]
  We propose a large multi-modal diffusion transformer, dubbed as Diffusion Transformer Policy, to model continuous end-effector actions.<n>By leveraging the scaling capability of transformers, the proposed approach can effectively model continuous end-effector actions across large diverse robot datasets.
  arXiv  Detail & Related papers  (2024-10-21T12:43:54Z)
• Temporal-Spatial Neural Filter: Direction Informed End-to-End Multi-channel Target Speech Separation [66.46123655365113]
  Target speech separation refers to extracting the target speaker's speech from mixed signals. Two main challenges are the complex acoustic environment and the real-time processing requirement. We propose a temporal-spatial neural filter, which directly estimates the target speech waveform from multi-speaker mixture.
  arXiv  Detail & Related papers  (2020-01-02T11:12:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.