Reliability-Aware Geometric Fusion for Robust Audio-Visual Navigation

• URL: http://arxiv.org/abs/2604.02391v1
• Date: Thu, 02 Apr 2026 07:26:46 GMT
• Title: Reliability-Aware Geometric Fusion for Robust Audio-Visual Navigation
• Abstract summary: RAVN (Reliability-Aware Audio-Visual Navigation) is a framework that conditions cross-modal fusion on audio-derived reliability cues.<n>We introduce Acoustic Geometry Reasoner (AGR) that is trained with geometric proxy supervision.<n>We also introduce Reliability-Aware Geometric Modulation (RAGM) which converts the learned cue into a soft gate to modulate visual features.
• Score: 4.024922311680863
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Audio-Visual Navigation (AVN) requires an embodied agent to navigate toward a sound source by utilizing both vision and binaural audio. A core challenge arises in complex acoustic environments, where binaural cues become intermittently unreliable, particularly when generalizing to previously unheard sound categories. To address this, we propose RAVN (Reliability-Aware Audio-Visual Navigation), a framework that conditions cross-modal fusion on audio-derived reliability cues, dynamically calibrating the integration of audio and visual inputs. RAVN introduces an Acoustic Geometry Reasoner (AGR) that is trained with geometric proxy supervision. Using a heteroscedastic Gaussian NLL objective, AGR learns observation-dependent dispersion as a practical reliability cue, eliminating the need for geometric labels during inference. Additionally, we introduce Reliability-Aware Geometric Modulation (RAGM), which converts the learned cue into a soft gate to modulate visual features, thereby mitigating cross-modal conflicts. We evaluate RAVN on SoundSpaces using both Replica and Matterport3D environments, and the results show consistent improvements in navigation performance, with notable robustness in the challenging unheard sound setting.

Related papers

• Attention-Guided Reliability Scaling for Contrastive Decoding in Robust Audio-Visual Speech Recognition [50.15288513086986]
  Contrastive decoding (CD) originally introduced to stabilize large language model (LLM)-based audio-visual speech recognition (AVSR) systems.<n>We propose reliability-aware scaling of CD for AVSR.
  arXiv  Detail & Related papers  (2026-08-26T08:25:15Z)
• LLM-Empowered Multimodal Fusion Framework for Autonomous Driving: Semantic Enhancement and Channel-Adaptive Design [56.76240343283953]
  We propose a Large Language Model-centric Semantic-layer Channel-aware Integrated Perception framework.<n>It fuses a local visual stream with a quality-varying external radar stream used to cover perception-blind spots.<n>Experiments on nuScenes and VIRAT validate our approach.
  arXiv  Detail & Related papers  (2026-07-02T06:44:02Z)
• Dynamic Spectral Denoising with Global-Context Attention for Multi-Behavior Recommendation [58.33732779555114]
  We argue that the key bottleneck is a representation-level failure caused by two coupled heterogeneities.<n>We propose Dynamic Spectral Denoising with Global-Context Attention for Multi-Behavior Recommendation (SpectraMB)<n>SpectraMB achieves the best results in most evaluation settings and exhibits improved robustness under noisy interactions.
  arXiv  Detail & Related papers  (2026-06-01T15:58:13Z)
• Audio Spatially-Guided Fusion for Audio-Visual Navigation [6.536516025407856]
  We propose an Audio Spatially-Guided Fusion for Audio-Visual Navigation method.<n>First, we design an audio spatial feature encoder, which adaptively extracts target-related spatial state information.<n>Based on this, we introduce an Audio Spatial State Guided Fusion (ASGF) to achieve dynamic alignment and adaptive fusion of multimodal features.
  arXiv  Detail & Related papers  (2026-04-02T07:15:17Z)
• Explainable Transformer-CNN Fusion for Noise-Robust Speech Emotion Recognition [2.0391237204597363]
  Speech Emotion Recognition systems often degrade in performance when exposed to unpredictable acoustic interference.<n>We propose a Hybrid Transformer-CNN framework that unifies the contextual modeling of Wav2Vec 2.0 with the spectral stability of 1D-Convolutional Neural Networks.
  arXiv  Detail & Related papers  (2025-12-20T10:05:58Z)
• SPUR: A Plug-and-Play Framework for Integrating Spatial Audio Understanding and Reasoning into Large Audio-Language Models [62.14165748145729]
  We introduce SPUR, a lightweight, plug-in approach that equips large audio-speaker models with spatial perception.<n>SPUR consists of: (i) a First-Order Ambisonics (FOA) encoder that maps channels to rotation-aware, listener-centric spatial features, integrated into target LALMs via a multimodal adapter; and (ii) SPUR-Set, a spatial QA dataset combining open-source FOA recordings with controlled simulations, emphasizing relative direction, elevation, distance, and overlap for supervised spatial reasoning.
  arXiv  Detail & Related papers  (2025-11-10T01:29:26Z)
• Audio-Guided Dynamic Modality Fusion with Stereo-Aware Attention for Audio-Visual Navigation [41.85539404067887]
  In audio-visual navigation (AVN) tasks, an embodied agent must autonomously localize a sound source in complex 3D environments.<n>Existing methods often rely on static modality fusion strategies and neglect the spatial cues embedded in stereo audio.<n>We propose an end-to-end reinforcement learning-based AVN framework with two key innovations.
  arXiv  Detail & Related papers  (2025-09-21T05:11:09Z)
• Improving Noise Robust Audio-Visual Speech Recognition via Router-Gated Cross-Modal Feature Fusion [46.072071890391356]
  We propose a novel framework that adaptively reweights audio and visual features based on token-level acoustic corruption scores.<n>Using an audio-visual feature fusion-based router, our method down-weights unreliable audio tokens and reinforces visual cues through cross-attention gated in each decoder layer.<n> Experiments on LRS3 demonstrate that our approach achieves an 16.51-42.67% relative reduction in word error rate compared to AV-HuBERT.
  arXiv  Detail & Related papers  (2025-08-26T07:05:48Z)
• Adaptive Control Attention Network for Underwater Acoustic Localization and Domain Adaptation [8.017203108408973]
  Localizing acoustic sound sources in the ocean is a challenging task due to the complex and dynamic nature of the environment.<n>We propose a multi-branch network architecture designed to accurately predict the distance between a moving acoustic source and a receiver.<n>Our proposed method outperforms state-of-the-art (SOTA) approaches in similar settings.
  arXiv  Detail & Related papers  (2025-06-20T18:13:30Z)
• AV-GS: Learning Material and Geometry Aware Priors for Novel View Acoustic Synthesis [62.33446681243413]
  view acoustic synthesis aims to render audio at any target viewpoint, given a mono audio emitted by a sound source at a 3D scene.<n>Existing methods have proposed NeRF-based implicit models to exploit visual cues as a condition for synthesizing audio.<n>We propose a novel Audio-Visual Gaussian Splatting (AV-GS) model to characterize the entire scene environment.<n>Experiments validate the superiority of our AV-GS over existing alternatives on the real-world RWAS and simulation-based SoundSpaces datasets.
  arXiv  Detail & Related papers  (2024-06-13T08:34:12Z)
• Adaptive ship-radiated noise recognition with learnable fine-grained wavelet transform [25.887932248706218]
  This work proposes an adaptive generalized recognition system - AGNet. By converting fixed wavelet parameters into fine-grained learnable parameters, AGNet learns the characteristics of underwater sound at different frequencies. Experiments reveal that our AGNet outperforms all baseline methods on several underwater acoustic datasets.
  arXiv  Detail & Related papers  (2023-05-31T06:56:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.