Advancing Audio-Visual Navigation Through Multi-Agent Collaboration in 3D Environments

• URL: http://arxiv.org/abs/2509.22698v1
• Date: Sun, 21 Sep 2025 05:05:26 GMT
• Title: Advancing Audio-Visual Navigation Through Multi-Agent Collaboration in 3D Environments
• Abstract summary: MASTAVN is a scalable framework enabling two agents to collaboratively localize and navigate toward an audio target in shared 3D environments.<n>MASTAVN achieves significant reductions in task completion time and notable improvements in navigation success rates compared to single-agent and non-collaborative baselines.
• Score: 37.73614419658084
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Intelligent agents often require collaborative strategies to achieve complex tasks beyond individual capabilities in real-world scenarios. While existing audio-visual navigation (AVN) research mainly focuses on single-agent systems, their limitations emerge in dynamic 3D environments where rapid multi-agent coordination is critical, especially for time-sensitive applications like emergency response. This paper introduces MASTAVN (Multi-Agent Scalable Transformer Audio-Visual Navigation), a scalable framework enabling two agents to collaboratively localize and navigate toward an audio target in shared 3D environments. By integrating cross-agent communication protocols and joint audio-visual fusion mechanisms, MASTAVN enhances spatial reasoning and temporal synchronization. Through rigorous evaluation in photorealistic 3D simulators (Replica and Matterport3D), MASTAVN achieves significant reductions in task completion time and notable improvements in navigation success rates compared to single-agent and non-collaborative baselines. This highlights the essential role of spatiotemporal coordination in multi-agent systems. Our findings validate MASTAVN's effectiveness in time-sensitive emergency scenarios and establish a paradigm for advancing scalable multi-agent embodied intelligence in complex 3D environments.

Related papers

• Gamma-World: Generative Multi-Agent World Modeling Beyond Two Players [87.87842088508553]
  We present our generative multi-agent world model for interactive simulation.<n>Our model improves video fidelity, action controllability, and inter-agent consistency over slot-based and dense-attention baselines.
  arXiv  Detail & Related papers  (2026-05-27T17:59:31Z)
• DeCoNav: Dialog enhanced Long-Horizon Collaborative Vision-Language Navigation [51.978605314711835]
  Long-horizon collaborative vision-language navigation (VLN) is critical for multi-robot systems to accomplish complex tasks beyond the capability of a single agent.<n>We present Dialog enhanced Long-Horizon Collaborative Vision-Language Navigation (DeCoNav)<n>DeCoNav is a decentralized framework that couples event-triggered dialogue with dynamic task allocation and replanning for real-time, adaptive coordination.
  arXiv  Detail & Related papers  (2026-04-14T09:11:55Z)
• CATNet: Collaborative Alignment and Transformation Network for Cooperative Perception [9.983779569276475]
  Collaborative Alignment and Transformation Network (CATNet) is an adaptive compensation framework that resolves temporal latency and noise interference in multi-agent systems.<n>Our key innovations can be summarized in three aspects. First, we introduce a Spatio-Temporal Recurrent Synchronization (STSync) that aligns asynchronous feature streams.<n>Second, we design a Dual-Branch Wavelet Enhanced Denoiser (WTDen) that suppresses global noise and reconstructs localized feature distortions.<n>Third, we construct an Adaptive Feature Selector (AdpSel) that dynamically focuses on critical perceptual features for robust fusion.
  arXiv  Detail & Related papers  (2026-03-05T15:07:36Z)
• Diffusion Forcing for Multi-Agent Interaction Sequence Modeling [52.769202433667125]
  MAGNet is a unified autoregressive diffusion framework for multi-agent motion generation.<n>It supports a wide range of interaction tasks through flexible conditioning and sampling.<n>It captures both tightly synchronized activities and loosely structured social interactions.
  arXiv  Detail & Related papers  (2025-12-19T18:59:02Z)
• AMUSE: Audio-Visual Benchmark and Alignment Framework for Agentic Multi-Speaker Understanding [73.05946667683259]
  Recent large language models (MLLMs) show strong perception but struggle in multi-speaker, dialogue-centric settings.<n>We introduce AMUSE, a benchmark designed around tasks that are inherently agentic.<n>We propose RAFT, a data-efficient agentic alignment framework that integrates reward optimization with intrinsic multimodal self-evaluation.
  arXiv  Detail & Related papers  (2025-12-18T07:01:47Z)
• TP-MDDN: Task-Preferenced Multi-Demand-Driven Navigation with Autonomous Decision-Making [90.18833928208333]
  Task-Preferenced Multi-Demand-Driven Navigation (TP-MDDN) is a new benchmark for long-horizon navigation involving multiple sub-demands with explicit task preferences.<n>For spatial memory, we design MASMap, which combines 3D point cloud accumulation with 2D semantic mapping for accurate and efficient environmental understanding.<n>Our approach outperforms state-of-the-art baselines in both perception accuracy and navigation robustness.
  arXiv  Detail & Related papers  (2025-11-21T13:12:13Z)
• Efficient Multi-Person Motion Prediction by Lightweight Spatial and Temporal Interactions [45.51160285910023]
  We propose a computationally efficient model for multi-person motion prediction by simplifying spatial and temporal interactions.<n>We achieve state-of-the-art performance for multiple metrics on standard datasets of CMU-Mocap, MuPoTS-3D, and 3DPW.
  arXiv  Detail & Related papers  (2025-07-13T02:16:37Z)
• Which2comm: An Efficient Collaborative Perception Framework for 3D Object Detection [5.195291754828701]
  Collaborative perception allows real-time inter-agent information exchange.<n>limited communication bandwidth in practical scenarios restricts the inter-agent data transmission volume.<n>We propose Which2comm, a novel multi-agent 3D object detection framework leveraging object-level sparse features.
  arXiv  Detail & Related papers  (2025-03-21T14:24:07Z)
• CSAOT: Cooperative Multi-Agent System for Active Object Tracking [1.384468678066823]
  Active Object Tracking (AOT) requires a controller agent to actively adjust its viewpoint to maintain visual contact with a moving target in complex environments.<n>Existing AOT solutions are predominantly single-agent-based, which struggle in dynamic and complex scenarios.<n>We introduce the Collaborative System for Active Object Tracking (CSAOT) to enable multiple agents to operate on a single device.
  arXiv  Detail & Related papers  (2025-01-23T10:44:35Z)
• UniTR: A Unified and Efficient Multi-Modal Transformer for Bird's-Eye-View Representation [113.35352122662752]
  We present an efficient multi-modal backbone for outdoor 3D perception named UniTR. UniTR processes a variety of modalities with unified modeling and shared parameters. UniTR is also a fundamentally task-agnostic backbone that naturally supports different 3D perception tasks.
  arXiv  Detail & Related papers  (2023-08-15T12:13:44Z)
• Spatial-Temporal Graph Enhanced DETR Towards Multi-Frame 3D Object Detection [54.041049052843604]
  We present STEMD, a novel end-to-end framework that enhances the DETR-like paradigm for multi-frame 3D object detection. First, to model the inter-object spatial interaction and complex temporal dependencies, we introduce the spatial-temporal graph attention network. Finally, it poses a challenge for the network to distinguish between the positive query and other highly similar queries that are not the best match.
  arXiv  Detail & Related papers  (2023-07-01T13:53:14Z)
• Ret3D: Rethinking Object Relations for Efficient 3D Object Detection in Driving Scenes [82.4186966781934]
  We introduce a simple, efficient, and effective two-stage detector, termed as Ret3D. At the core of Ret3D is the utilization of novel intra-frame and inter-frame relation modules. With negligible extra overhead, Ret3D achieves the state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-08-18T03:48:58Z)
• Searching Multi-Rate and Multi-Modal Temporal Enhanced Networks for Gesture Recognition [89.0152015268929]
  We propose the first neural architecture search (NAS)-based method for RGB-D gesture recognition. The proposed method includes two key components: 1) enhanced temporal representation via the 3D Central Difference Convolution (3D-CDC) family, and optimized backbones for multi-modal-rate branches and lateral connections. The resultant multi-rate network provides a new perspective to understand the relationship between RGB and depth modalities and their temporal dynamics.
  arXiv  Detail & Related papers  (2020-08-21T10:45:09Z)
• A Visual Communication Map for Multi-Agent Deep Reinforcement Learning [7.003240657279981]
  Multi-agent learning poses significant challenges in the effort to allocate a concealed communication medium. Recent studies typically combine a specialized neural network with reinforcement learning to enable communication between agents. This paper proposes a more scalable approach that not only deals with a great number of agents but also enables collaboration between dissimilar functional agents.
  arXiv  Detail & Related papers  (2020-02-27T02:38:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.