Robust Deep Learning from Crowds with Belief Propagation
- URL: http://arxiv.org/abs/2111.00734v1
- Date: Mon, 1 Nov 2021 07:20:16 GMT
- Title: Robust Deep Learning from Crowds with Belief Propagation
- Authors: Hoyoung Kim, Seunghyuk Cho, Dongwoo Kim, Jungseul Ok
- Abstract summary: A graphical model representing local dependencies between workers and tasks provides a principled way of reasoning over the true labels from the noisy answers.
One needs a predictive model working on unseen data directly from crowdsourced datasets instead of the true labels in many cases.
We propose a new data-generating process, where a neural network generates the true labels from task features.
- Score: 6.643082745560235
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Crowdsourcing systems enable us to collect noisy labels from crowd workers. A
graphical model representing local dependencies between workers and tasks
provides a principled way of reasoning over the true labels from the noisy
answers. However, one needs a predictive model working on unseen data directly
from crowdsourced datasets instead of the true labels in many cases. To infer
true labels and learn a predictive model simultaneously, we propose a new
data-generating process, where a neural network generates the true labels from
task features. We devise an EM framework alternating variational inference and
deep learning to infer the true labels and to update the neural network,
respectively. Experimental results with synthetic and real datasets show a
belief-propagation-based EM algorithm is robust to i) corruption in task
features, ii) multi-modal or mismatched worker prior, and iii) few spammers
submitting noises to many tasks.
Related papers
- Jump-teaching: Ultra Efficient and Robust Learning with Noisy Label [6.818488262543482]
We propose a novel technique to distinguish mislabeled samples during training.
We employ only one network with the jump manner update to decouple the interplay and mine more semantic information from the loss.
Our proposed approach achieves almost up to $2.53times$ speedup, $0.56times$ peak memory footprint, and superior robustness over state-of-the-art works with various noise settings.
arXiv Detail & Related papers (2024-05-27T12:54:09Z) - Learning with Noisy Labels: Interconnection of Two
Expectation-Maximizations [41.65589788264123]
Labor-intensive labeling becomes a bottleneck in developing computer vision algorithms based on deep learning.
We address learning with noisy labels (LNL) problem, which is formalized as a task of finding a structured manifold in the midst of noisy data.
Our algorithm achieves state-of-the-art performance in multiple standard benchmarks with substantial margins under various types of label noise.
arXiv Detail & Related papers (2024-01-09T07:22:30Z) - Label Selection Approach to Learning from Crowds [25.894399244406287]
Learning from Crowds is a framework which directly trains the models using noisy labeled data from crowd workers.
We propose a novel Learning from Crowds model, inspired by SelectiveNet proposed for the selective prediction problem.
A major advantage of the proposed method is that it can be applied to almost all variants of supervised learning problems.
arXiv Detail & Related papers (2023-08-21T00:22:32Z) - Label-Retrieval-Augmented Diffusion Models for Learning from Noisy
Labels [61.97359362447732]
Learning from noisy labels is an important and long-standing problem in machine learning for real applications.
In this paper, we reformulate the label-noise problem from a generative-model perspective.
Our model achieves new state-of-the-art (SOTA) results on all the standard real-world benchmark datasets.
arXiv Detail & Related papers (2023-05-31T03:01:36Z) - Learning to Detect Noisy Labels Using Model-Based Features [16.681748918518075]
We propose Selection-Enhanced Noisy label Training (SENT)
SENT does not rely on meta learning while having the flexibility of being data-driven.
It improves performance over strong baselines under the settings of self-training and label corruption.
arXiv Detail & Related papers (2022-12-28T10:12:13Z) - Instance-dependent Label-noise Learning under a Structural Causal Model [92.76400590283448]
Label noise will degenerate the performance of deep learning algorithms.
By leveraging a structural causal model, we propose a novel generative approach for instance-dependent label-noise learning.
arXiv Detail & Related papers (2021-09-07T10:42:54Z) - Tackling Instance-Dependent Label Noise via a Universal Probabilistic
Model [80.91927573604438]
This paper proposes a simple yet universal probabilistic model, which explicitly relates noisy labels to their instances.
Experiments on datasets with both synthetic and real-world label noise verify that the proposed method yields significant improvements on robustness.
arXiv Detail & Related papers (2021-01-14T05:43:51Z) - Noisy Labels Can Induce Good Representations [53.47668632785373]
We study how architecture affects learning with noisy labels.
We show that training with noisy labels can induce useful hidden representations, even when the model generalizes poorly.
This finding leads to a simple method to improve models trained on noisy labels.
arXiv Detail & Related papers (2020-12-23T18:58:05Z) - Exploiting Heterogeneous Graph Neural Networks with Latent Worker/Task
Correlation Information for Label Aggregation in Crowdsourcing [72.34616482076572]
Crowdsourcing has attracted much attention for its convenience to collect labels from non-expert workers instead of experts.
We propose a novel framework based on graph neural networks for aggregating crowd labels.
arXiv Detail & Related papers (2020-10-25T10:12:37Z) - Low Resource Multi-Task Sequence Tagging -- Revisiting Dynamic
Conditional Random Fields [67.51177964010967]
We compare different models for low resource multi-task sequence tagging that leverage dependencies between label sequences for different tasks.
We find that explicit modeling of inter-dependencies between task predictions outperforms single-task as well as standard multi-task models.
arXiv Detail & Related papers (2020-05-01T07:11:34Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.