L3DAS21 Challenge: Machine Learning for 3D Audio Signal Processing

• URL: http://arxiv.org/abs/2104.05499v1
• Date: Mon, 12 Apr 2021 14:29:54 GMT
• Title: L3DAS21 Challenge: Machine Learning for 3D Audio Signal Processing
• Authors: Eric Guizzo, Riccardo F. Gramaccioni, Saeid Jamili, Christian Marinoni, Edoardo Massaro, Claudia Medaglia, Giuseppe Nachira, Leonardo Nucciarelli, Ludovica Paglialunga, Marco Pennese, Sveva Pepe, Enrico Rocchi, Aurelio Uncini, Danilo Comminiello
• Abstract summary: The L3DAS21 Challenge is aimed at encouraging and fostering collaborative research on machine learning for 3D audio signal processing. We release the L3DAS21 dataset, a 65 hours 3D audio corpus, accompanied with a Python API that facilitates the data usage and results submission stage.
• Score: 6.521891605165917
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The L3DAS21 Challenge is aimed at encouraging and fostering collaborative research on machine learning for 3D audio signal processing, with particular focus on 3D speech enhancement (SE) and 3D sound localization and detection (SELD). Alongside with the challenge, we release the L3DAS21 dataset, a 65 hours 3D audio corpus, accompanied with a Python API that facilitates the data usage and results submission stage. Usually, machine learning approaches to 3D audio tasks are based on single-perspective Ambisonics recordings or on arrays of single-capsule microphones. We propose, instead, a novel multichannel audio configuration based multiple-source and multiple-perspective Ambisonics recordings, performed with an array of two first-order Ambisonics microphones. To the best of our knowledge, it is the first time that a dual-mic Ambisonics configuration is used for these tasks. We provide baseline models and results for both tasks, obtained with state-of-the-art architectures: FaSNet for SE and SELDNet for SELD. This report is aimed at providing all needed information to participate in the L3DAS21 Challenge, illustrating the details of the L3DAS21 dataset, the challenge tasks and the baseline models.

Related papers

• Meerkat: Audio-Visual Large Language Model for Grounding in Space and Time [73.7845280328535]
  We present Meerkat, an audio-visual LLM equipped with a fine-grained understanding of image and audio. Meerkat can tackle challenging tasks such as audio referred image grounding, image guided audio temporal localization, and audio-visual fact-checking. We achieve state-of-the-art performance on all these downstream tasks with a relative improvement of up to 37.12%.
  arXiv  Detail & Related papers  (2024-07-01T23:32:25Z)
• The NeurIPS 2023 Machine Learning for Audio Workshop: Affective Audio Benchmarks and Novel Data [28.23517306589778]
  The NeurIPS 2023 Machine Learning for Audio Workshop brings together machine learning (ML) experts from various audio domains. There are several valuable audio-driven ML tasks, from speech emotion recognition to audio event detection, but the community is sparse compared to other ML areas. High-quality data collection is time-consuming and costly, making it challenging for academic groups to apply their often state-of-the-art strategies to a larger, more generalizable dataset.
  arXiv  Detail & Related papers  (2024-03-21T00:13:59Z)
• Overview of the L3DAS23 Challenge on Audio-Visual Extended Reality [15.034352805342937]
  The primary goal of the L3DAS23 Signal Processing Grand Challenge at ICASSP 2023 is to promote and support collaborative research on machine learning for 3D audio signal processing. We provide a brand-new dataset, which maintains the same general characteristics of the L3DAS21 and L3DAS22 datasets. We propose updated baseline models for both tasks that can now support audio-image couples as input and a supporting API to replicate our results.
  arXiv  Detail & Related papers  (2024-02-14T15:34:28Z)
• Novel-View Acoustic Synthesis from 3D Reconstructed Rooms [18.49261985372842]
  We investigate the benefit of combining blind audio recordings with 3D scene information for novel-view acoustic synthesis. We identify the main challenges of novel-view acoustic synthesis as sound source localization, separation, and dereverberation. We show that incorporating room impulse responses (RIRs) derived from 3D reconstructed rooms enables the same network to jointly tackle these tasks.
  arXiv  Detail & Related papers  (2023-10-23T17:34:31Z)
• Large-scale unsupervised audio pre-training for video-to-speech synthesis [64.86087257004883]
  Video-to-speech synthesis is the task of reconstructing the speech signal from a silent video of a speaker. In this paper we propose to train encoder-decoder models on more than 3,500 hours of audio data at 24kHz. We then use the pre-trained decoders to initialize the audio decoders for the video-to-speech synthesis task.
  arXiv  Detail & Related papers  (2023-06-27T13:31:33Z)
• SVTS: Scalable Video-to-Speech Synthesis [105.29009019733803]
  We introduce a scalable video-to-speech framework consisting of two components: a video-to-spectrogram predictor and a pre-trained neural vocoder. We are the first to show intelligible results on the challenging LRS3 dataset.
  arXiv  Detail & Related papers  (2022-05-04T13:34:07Z)
• DSGN++: Exploiting Visual-Spatial Relation forStereo-based 3D Detectors [60.88824519770208]
  Camera-based 3D object detectors are welcome due to their wider deployment and lower price than LiDAR sensors. We revisit the prior stereo modeling DSGN about the stereo volume constructions for representing both 3D geometry and semantics. We propose our approach, DSGN++, aiming for improving information flow throughout the 2D-to-3D pipeline.
  arXiv  Detail & Related papers  (2022-04-06T18:43:54Z)
• L3DAS22 Challenge: Learning 3D Audio Sources in a Real Office Environment [12.480610577162478]
  The L3DAS22 Challenge is aimed at encouraging the development of machine learning strategies for 3D speech enhancement and 3D sound localization and detection. This challenge improves and extends the tasks of the L3DAS21 edition.
  arXiv  Detail & Related papers  (2022-02-21T17:05:39Z)
• Sound and Visual Representation Learning with Multiple Pretraining Tasks [104.11800812671953]
  Self-supervised tasks (SSL) reveal different features from the data. This work aims to combine Multiple SSL tasks (Multi-SSL) that generalizes well for all downstream tasks. Experiments on sound representations demonstrate that Multi-SSL via incremental learning (IL) of SSL tasks outperforms single SSL task models.
  arXiv  Detail & Related papers  (2022-01-04T09:09:38Z)
• Semantic Object Prediction and Spatial Sound Super-Resolution with Binaural Sounds [106.87299276189458]
  Humans can robustly recognize and localize objects by integrating visual and auditory cues. This work develops an approach for dense semantic labelling of sound-making objects, purely based on sounds.
  arXiv  Detail & Related papers  (2020-03-09T15:49:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.