Steering Deep Non-Linear Spatially Selective Filters for Weakly Guided Extraction of Moving Speakers in Dynamic Scenarios

• URL: http://arxiv.org/abs/2505.14517v1
• Date: Tue, 20 May 2025 15:43:55 GMT
• Title: Steering Deep Non-Linear Spatially Selective Filters for Weakly Guided Extraction of Moving Speakers in Dynamic Scenarios
• Abstract summary: spatially dynamic scenarios are considerably more challenging due to time-varying spatial features and arising ambiguities.<n>We propose a weakly guided extraction method solely depending on the target's initial position to cope with spatial dynamic scenarios.<n>By incorporating our own deep tracking algorithm and developing a joint training strategy on a synthetic dataset, we demonstrate the proficiency of our approach in resolving spatial ambiguities.
• Score: 15.736484513462973
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent speaker extraction methods using deep non-linear spatial filtering perform exceptionally well when the target direction is known and stationary. However, spatially dynamic scenarios are considerably more challenging due to time-varying spatial features and arising ambiguities, e.g. when moving speakers cross. While in a static scenario it may be easy for a user to point to the target's direction, manually tracking a moving speaker is impractical. Instead of relying on accurate time-dependent directional cues, which we refer to as strong guidance, in this paper we propose a weakly guided extraction method solely depending on the target's initial position to cope with spatial dynamic scenarios. By incorporating our own deep tracking algorithm and developing a joint training strategy on a synthetic dataset, we demonstrate the proficiency of our approach in resolving spatial ambiguities and even outperform a mismatched, but strongly guided extraction method.

Related papers

• AutoPath: Learning Transferable Goal-Conditioned Stochastic Path Prior for Safe Navigation Without Human Demonstrations [103.61864552260316]
  Real-time navigation in dynamic environments requires collision-free and dynamically feasible motion under limited perception.<n>We formulate navigation as learning a transferable goal-aligned path prior that models a reusable distribution over goal-aligned geometry-consistent local paths conditioned on local observations.<n>We develop a structured prior learning framework that parameterizes local paths using a geometry-aware polar action manifold and incorporates risk-sensitive utility shaping with multi-goal distributional rollouts for stable and safety-aware planning.
  arXiv  Detail & Related papers  (2026-07-13T16:00:41Z)
• SelectTSL: Prompt-Guided Selective Target Sound Localization in Complex Scenarios [95.0451976922633]
  We propose an end-to-end architecture that localizes only the user-specified target in multi-source acoustic scenes.<n>Specifically, we design a target-aware selective localization strategy that employs a Prompt-Guided Selective Attention Module.<n>This coupled design effectively focuses on the user-specified target spatial cues for selective localization and also handles time-varying numbers of target sources.
  arXiv  Detail & Related papers  (2026-07-02T15:49:10Z)
• Active Timepoint Selection for Learning Measure-Valued Trajectories [55.49809248697036]
  Inferring continuous probability paths from snapshots is a fundamental challenge in domains like single-cell biology.<n>We introduce a framework which extends active experimentation to the space of sparse measures.<n>This yields an acquisition policy that iteratively selects measurement times to minimize uncertainty.
  arXiv  Detail & Related papers  (2026-05-28T22:22:35Z)
• Referring-Aware Visuomotor Policy Learning for Closed-Loop Manipulation [91.20850436220267]
  We introduce the Referring-Aware Visuomotor Policy (ReV)<n>ReV incorporates sparse referring points provided by a human or a high-level reasoning planner.<n>It is trained only by applying targeted perturbations to expert demonstrations.
  arXiv  Detail & Related papers  (2026-04-07T07:41:11Z)
• Autoregressive Guidance of Deep Spatially Selective Filters using Bayesian Tracking for Efficient Extraction of Moving Speakers [28.32055656460997]
  In this work, we investigate strategies to incorporate the enhanced signal into lightweight tracking algorithms and autoregressively guide deep spatial filters.<n>Results validate that the autoregressive incorporation significantly improves the accuracy of our Bayesian trackers, resulting in superior enhancement with none or only negligibly increased computational overhead.
  arXiv  Detail & Related papers  (2026-03-24T21:19:45Z)
• Streaming Real-Time Trajectory Prediction Using Endpoint-Aware Modeling [54.94692733670454]
  Future trajectories of neighboring traffic agents have a significant influence on the path planning and decision-making of autonomous vehicles.<n>We propose a lightweight yet highly accurate streaming-based trajectory forecasting approach.<n>Our approach significantly reduces inference latency, making it well-suited for real-world deployment.
  arXiv  Detail & Related papers  (2026-03-02T13:44:23Z)
• RSGround-R1: Rethinking Remote Sensing Visual Grounding through Spatial Reasoning [61.84363374647606]
  Remote Sensing Visual Grounding (RSVG) aims to localize target objects in large-scale aerial imagery based on natural language descriptions.<n>These descriptions often rely heavily on positional cues, posing unique challenges for Multimodal Large Language Models (MLLMs) in spatial reasoning.<n>We propose a reasoning-guided, position-aware post-training framework, dubbed textbfRSGround-R1, to progressively enhance spatial understanding.
  arXiv  Detail & Related papers  (2026-01-29T12:35:57Z)
• Adaptive Rotary Steering with Joint Autoregression for Robust Extraction of Closely Moving Speakers in Dynamic Scenarios [28.32055656460997]
  Latest advances in deep spatial filtering for Ambisonics demonstrate strong performance in stationary multi-speaker scenarios.<n>We propose to automate this rotary steering using an interleaved tracking algorithm conditioned on the target's initial direction.<n>For nearby or crossing speakers, robust tracking becomes difficult and spatial cues less effective for enhancement.
  arXiv  Detail & Related papers  (2026-01-18T10:38:49Z)
• Learning Multi-Modal Mobility Dynamics for Generalized Next Location Recommendation [51.00494428978262]
  We leverage multi-modal spatial-temporal knowledge to characterize mobility dynamics for the location recommendation task.<n>First, we construct a unified spatial-temporal relational graph (STRG) for multi-modal representation.<n>Second, we design a gating mechanism to fuse spatial-temporal graph representations of different modalities.
  arXiv  Detail & Related papers  (2025-12-27T14:23:04Z)
• Self-Steering Deep Non-Linear Spatially Selective Filters for Efficient Extraction of Moving Speakers under Weak Guidance [14.16697537117357]
  We present a novel strategy utilizing a low-complexity tracking algorithm in the form of a particle filter instead.<n>We show how the autoregressive interplay between both algorithms drastically improves tracking accuracy and leads to strong enhancement performance.
  arXiv  Detail & Related papers  (2025-07-03T16:54:56Z)
• Seurat: From Moving Points to Depth [66.65189052568209]
  We propose a novel method that infers relative depth by examining the spatial relationships and temporal evolution of a set of tracked 2D trajectories.<n>Our approach achieves temporally smooth, high-accuracy depth predictions across diverse domains.
  arXiv  Detail & Related papers  (2025-04-20T17:37:02Z)
• Learning Where to Look: Self-supervised Viewpoint Selection for Active Localization using Geometrical Information [68.10033984296247]
  This paper explores the domain of active localization, emphasizing the importance of viewpoint selection to enhance localization accuracy. Our contributions involve using a data-driven approach with a simple architecture designed for real-time operation, a self-supervised data training method, and the capability to consistently integrate our map into a planning framework tailored for real-world robotics applications.
  arXiv  Detail & Related papers  (2024-07-22T12:32:09Z)
• Self-Supervised Class-Agnostic Motion Prediction with Spatial and Temporal Consistency Regularizations [53.797896854533384]
  Class-agnostic motion prediction methods directly predict the motion of the entire point cloud. While most existing methods rely on fully-supervised learning, the manual labeling of point cloud data is laborious and time-consuming. We introduce three simple spatial and temporal regularization losses, which facilitate the self-supervised training process effectively.
  arXiv  Detail & Related papers  (2024-03-20T02:58:45Z)
• Attention-Driven Multichannel Speech Enhancement in Moving Sound Source Scenarios [11.811571392419324]
  Speech enhancement algorithms typically assume a stationary sound source, a common mismatch with reality that limits their performance in real-world scenarios. This paper focuses on attention-driven spatial filtering techniques designed for dynamic settings.
  arXiv  Detail & Related papers  (2023-12-17T16:12:35Z)
• Learning Representative Trajectories of Dynamical Systems via Domain-Adaptive Imitation [0.0]
  We propose DATI, a deep reinforcement learning agent designed for domain-adaptive trajectory imitation. Our experiments show that DATI outperforms baseline methods for imitation learning and optimal control in this setting. Its generalization to a real-world scenario is shown through the discovery of abnormal motion patterns in maritime traffic.
  arXiv  Detail & Related papers  (2023-04-19T15:53:48Z)
• Spatially Selective Deep Non-linear Filters for Speaker Extraction [21.422488450492434]
  We develop a deep joint spatial-spectral non-linear filter that can be steered in an arbitrary target direction. We show that this scheme is more effective than the baseline approach and increases the flexibility of the filter at no performance cost.
  arXiv  Detail & Related papers  (2022-11-04T12:54:06Z)
• Pre-training General Trajectory Embeddings with Maximum Multi-view Entropy Coding [36.18788551389281]
  Trajectory embeddings can improve task performance but may incur high computational costs and face limited training data availability. Existing trajectory embedding methods face difficulties in learning general embeddings due to biases towards certain downstream tasks. We propose Multi-view Trajectory Entropy Coding Coding (MMTEC) for learning general comprehensive trajectory embeddings.
  arXiv  Detail & Related papers  (2022-07-29T08:16:20Z)
• Deep Shells: Unsupervised Shape Correspondence with Optimal Transport [52.646396621449]
  We propose a novel unsupervised learning approach to 3D shape correspondence. We show that the proposed method significantly improves over the state-of-the-art on multiple datasets.
  arXiv  Detail & Related papers  (2020-10-28T22:24:07Z)
• AutoTrajectory: Label-free Trajectory Extraction and Prediction from Videos using Dynamic Points [92.91569287889203]
  We present a novel, label-free algorithm, AutoTrajectory, for trajectory extraction and prediction. To better capture the moving objects in videos, we introduce dynamic points. We aggregate dynamic points to instance points, which stand for moving objects such as pedestrians in videos.
  arXiv  Detail & Related papers  (2020-07-11T08:43:34Z)
• Temporal-Spatial Neural Filter: Direction Informed End-to-End Multi-channel Target Speech Separation [66.46123655365113]
  Target speech separation refers to extracting the target speaker's speech from mixed signals. Two main challenges are the complex acoustic environment and the real-time processing requirement. We propose a temporal-spatial neural filter, which directly estimates the target speech waveform from multi-speaker mixture.
  arXiv  Detail & Related papers  (2020-01-02T11:12:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.