ARCA23K: An audio dataset for investigating open-set label noise

• URL: http://arxiv.org/abs/2109.09227v1
• Date: Sun, 19 Sep 2021 21:10:25 GMT
• Title: ARCA23K: An audio dataset for investigating open-set label noise
• Authors: Turab Iqbal, Yin Cao, Andrew Bailey, Mark D. Plumbley, Wenwu Wang
• Abstract summary: This paper introduces ARCA23K, an automatically Retrieved and curated audio dataset comprised of over 23000 labelled Freesound clips. We show that the majority of labelling errors in ARCA23K are due to out-of-vocabulary audio clips, and we refer to this type of label noise as open-set label noise.
• Score: 48.683197172795865
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: The availability of audio data on sound sharing platforms such as Freesound gives users access to large amounts of annotated audio. Utilising such data for training is becoming increasingly popular, but the problem of label noise that is often prevalent in such datasets requires further investigation. This paper introduces ARCA23K, an Automatically Retrieved and Curated Audio dataset comprised of over 23000 labelled Freesound clips. Unlike past datasets such as FSDKaggle2018 and FSDnoisy18K, ARCA23K facilitates the study of label noise in a more controlled manner. We describe the entire process of creating the dataset such that it is fully reproducible, meaning researchers can extend our work with little effort. We show that the majority of labelling errors in ARCA23K are due to out-of-vocabulary audio clips, and we refer to this type of label noise as open-set label noise. Experiments are carried out in which we study the impact of label noise in terms of classification performance and representation learning.

Related papers

• Sound Check: Auditing Audio Datasets [4.955141080136429]
  Generative audio models are rapidly advancing in both capabilities and public utilization. We conducted a literature review of hundreds of audio datasets and selected seven of the most prominent to audit. We found that these datasets are biased against women, contain toxic stereotypes about marginalized communities, and contain significant amounts of copyrighted work.
  arXiv  Detail & Related papers  (2024-10-17T00:51:27Z)
• AlleNoise: large-scale text classification benchmark dataset with real-world label noise [40.11095094521714]
  We present AlleNoise, a new curated text classification benchmark dataset with real-world instance-dependent label noise. The noise distribution comes from actual users of a major e-commerce marketplace, so it realistically reflects the semantics of human mistakes. We demonstrate that a representative selection of established methods for learning with noisy labels is inadequate to handle such real-world noise.
  arXiv  Detail & Related papers  (2024-06-24T09:29:14Z)
• Handling Realistic Label Noise in BERT Text Classification [1.0515439489916731]
  Real label noise is not random; rather, it is often correlated with input features or other annotator-specific factors. We show that the presence of these types of noise significantly degrades BERT classification performance.
  arXiv  Detail & Related papers  (2023-05-23T18:30:31Z)
• Epic-Sounds: A Large-scale Dataset of Actions That Sound [64.24297230981168]
  Epic-Sounds is a large-scale dataset of audio annotations capturing temporal extents and class labels. We identify actions that can be discriminated purely from audio, through grouping these free-form descriptions of audio into classes. Overall, Epic-Sounds includes 78.4k categorised segments of audible events and actions, distributed across 44 classes as well as 39.2k non-categorised segments.
  arXiv  Detail & Related papers  (2023-02-01T18:19:37Z)
• S3: Supervised Self-supervised Learning under Label Noise [53.02249460567745]
  In this paper we address the problem of classification in the presence of label noise. In the heart of our method is a sample selection mechanism that relies on the consistency between the annotated label of a sample and the distribution of the labels in its neighborhood in the feature space. Our method significantly surpasses previous methods on both CIFARCIFAR100 with artificial noise and real-world noisy datasets such as WebVision and ANIMAL-10N.
  arXiv  Detail & Related papers  (2021-11-22T15:49:20Z)
• Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotations [54.400167806154535]
  Existing research on learning with noisy labels mainly focuses on synthetic label noise. This work presents two new benchmark datasets (CIFAR-10N, CIFAR-100N) We show that real-world noisy labels follow an instance-dependent pattern rather than the classically adopted class-dependent ones.
  arXiv  Detail & Related papers  (2021-10-22T22:42:11Z)
• A Second-Order Approach to Learning with Instance-Dependent Label Noise [58.555527517928596]
  The presence of label noise often misleads the training of deep neural networks. We show that the errors in human-annotated labels are more likely to be dependent on the difficulty levels of tasks.
  arXiv  Detail & Related papers  (2020-12-22T06:36:58Z)
• FSD50K: An Open Dataset of Human-Labeled Sound Events [30.42735806815691]
  We introduce FSD50K, an open dataset containing over 51k audio clips totalling over 100h of audio manually labeled using 200 classes drawn from the AudioSet Ontology. The audio clips are licensed under Creative Commons licenses, making the dataset freely distributable (including waveforms)
  arXiv  Detail & Related papers  (2020-10-01T15:07:25Z)
• Attention-Aware Noisy Label Learning for Image Classification [97.26664962498887]
  Deep convolutional neural networks (CNNs) learned on large-scale labeled samples have achieved remarkable progress in computer vision. The cheapest way to obtain a large body of labeled visual data is to crawl from websites with user-supplied labels, such as Flickr. This paper proposes the attention-aware noisy label learning approach to improve the discriminative capability of the network trained on datasets with potential label noise.
  arXiv  Detail & Related papers  (2020-09-30T15:45:36Z)
• Audio Tagging by Cross Filtering Noisy Labels [26.14064793686316]
  We present a novel framework, named CrossFilter, to combat the noisy labels problem for audio tagging. Our method achieves state-of-the-art performance and even surpasses the ensemble models.
  arXiv  Detail & Related papers  (2020-07-16T07:55:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.