SonoWorld: From One Image to a 3D Audio-Visual Scene

• URL: http://arxiv.org/abs/2603.28757v1
• Date: Mon, 30 Mar 2026 17:57:47 GMT
• Title: SonoWorld: From One Image to a 3D Audio-Visual Scene
• Abstract summary: We introduce Image2AVScene, the task of generating a 3D audio-visual scene from a single image.<n>We present SonoWorld, the first framework to tackle this challenge.<n>We also demonstrate applications to one-shot acoustic learning and audio-visual spatial source separation.
• Score: 29.64658395133738
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Tremendous progress in visual scene generation now turns a single image into an explorable 3D world, yet immersion remains incomplete without sound. We introduce Image2AVScene, the task of generating a 3D audio-visual scene from a single image, and present SonoWorld, the first framework to tackle this challenge. From one image, our pipeline outpaints a 360° panorama, lifts it into a navigable 3D scene, places language-guided sound anchors, and renders ambisonics for point, areal, and ambient sources, yielding spatial audio aligned with scene geometry and semantics. Quantitative evaluations on a newly curated real-world dataset and a controlled user study confirm the effectiveness of our approach. Beyond free-viewpoint audio-visual rendering, we also demonstrate applications to one-shot acoustic learning and audio-visual spatial source separation. Project website: https://humathe.github.io/sonoworld/

Related papers

• Scene2Sound: Auditory-Grounded Soundscape Generation for 3D Gaussian Worlds [51.55297978842271]
  3D Gaussian Splatting (3DGS) turns captured or generated imagery into 3D world simulations that users can freely explore, yet these worlds remain silent.<n>Because existing audio generation methods condition on a single image or viewpoint, their sound is tied to that observation and cannot stay consistent while a listener moves.<n>We introduce the task of generating a spatially consistent soundscape for a given 3DGS world through auditory grounding.<n>We present Scene2Sound, a training-free framework built on this grounding.
  arXiv  Detail & Related papers  (2026-08-01T06:04:34Z)
• WorldExplorer: Towards Generating Fully Navigable 3D Scenes [48.16064304951891]
  WorldExplorer builds fully navigable 3D scenes with consistent visual quality across a wide range of viewpoints.<n>We generate multiple videos along short, pre-defined trajectories, that explore the scene in depth.<n>Our novel scene memory conditions each video on the most relevant prior views, while a collision-detection mechanism prevents degenerate results.
  arXiv  Detail & Related papers  (2025-06-02T15:41:31Z)
• Visual Acoustic Fields [39.43953430861896]
  We propose Visual Acoustic Fields, a framework that bridges hitting sounds and visual signals within a 3D space.<n>Our approach features two key modules: sound generation and sound localization.<n>To the best of our knowledge, this is the first dataset to connect visual and acoustic signals in a 3D context.
  arXiv  Detail & Related papers  (2025-03-31T16:16:10Z)
• 3D Audio-Visual Segmentation [52.34970001474347]
  Recognizing the sounding objects in scenes is a longstanding objective in embodied AI, with diverse applications in robotics and AR/VR/MR.<n>We propose a new approach, EchoSegnet, characterized by integrating the ready-to-use knowledge from pretrained 2D audio-visual foundation models.<n>Experiments demonstrate that EchoSegnet can effectively segment sounding objects in 3D space on our new benchmark, representing a significant advancement in the field of embodied AI.
  arXiv  Detail & Related papers  (2024-11-04T16:30:14Z)
• AV-NeRF: Learning Neural Fields for Real-World Audio-Visual Scene Synthesis [61.07542274267568]
  We study a new task -- real-world audio-visual scene synthesis -- and a first-of-its-kind NeRF-based approach for multimodal learning. We propose an acoustic-aware audio generation module that integrates prior knowledge of audio propagation into NeRF. We present a coordinate transformation module that expresses a view direction relative to the sound source, enabling the model to learn sound source-centric acoustic fields.
  arXiv  Detail & Related papers  (2023-02-04T04:17:19Z)
• SceneDreamer: Unbounded 3D Scene Generation from 2D Image Collections [49.802462165826554]
  We present SceneDreamer, an unconditional generative model for unbounded 3D scenes. Our framework is learned from in-the-wild 2D image collections only, without any 3D annotations.
  arXiv  Detail & Related papers  (2023-02-02T18:59:16Z)
• Learning to Set Waypoints for Audio-Visual Navigation [89.42192208471735]
  In audio-visual navigation, an agent intelligently travels through a complex, unmapped 3D environment using both sights and sounds to find a sound source. Existing models learn to act at a fixed granularity of agent motion and rely on simple recurrent aggregations of the audio observations. We introduce a reinforcement learning approach to audio-visual navigation with two key novel elements.
  arXiv  Detail & Related papers  (2020-08-21T18:00:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.