KnowMAN: Weakly Supervised Multinomial Adversarial Networks
- URL: http://arxiv.org/abs/2109.07994v1
- Date: Thu, 16 Sep 2021 14:01:30 GMT
- Title: KnowMAN: Weakly Supervised Multinomial Adversarial Networks
- Authors: Luisa M\"arz, Ehsaneddin Asgari, Fabienne Braune, Franziska Zimmermann
and Benjamin Roth
- Abstract summary: We propose KnowMAN, an adversarial scheme that enables to control influence of signals associated with labeling functions.
KnowMAN improves results compared to direct weakly supervised learning with a pre-trained transformer language model and a feature-based baseline.
- Score: 8.135448729558876
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The absence of labeled data for training neural models is often addressed by
leveraging knowledge about the specific task, resulting in heuristic but noisy
labels. The knowledge is captured in labeling functions, which detect certain
regularities or patterns in the training samples and annotate corresponding
labels for training. This process of weakly supervised training may result in
an over-reliance on the signals captured by the labeling functions and hinder
models to exploit other signals or to generalize well. We propose KnowMAN, an
adversarial scheme that enables to control influence of signals associated with
specific labeling functions. KnowMAN forces the network to learn
representations that are invariant to those signals and to pick up other
signals that are more generally associated with an output label. KnowMAN
strongly improves results compared to direct weakly supervised learning with a
pre-trained transformer language model and a feature-based baseline.
Related papers
- Learning Discriminative Dynamics with Label Corruption for Noisy Label Detection [25.55455239006278]
We propose DynaCor framework that distinguishes incorrectly labeled instances from correctly labeled ones based on the dynamics of the training signals.
Our comprehensive experiments show that DynaCor outperforms the state-of-the-art competitors and shows strong robustness to various noise types and noise rates.
arXiv Detail & Related papers (2024-05-30T10:06:06Z) - Channel-Wise Contrastive Learning for Learning with Noisy Labels [60.46434734808148]
We introduce channel-wise contrastive learning (CWCL) to distinguish authentic label information from noise.
Unlike conventional instance-wise contrastive learning (IWCL), CWCL tends to yield more nuanced and resilient features aligned with the authentic labels.
Our strategy is twofold: firstly, using CWCL to extract pertinent features to identify cleanly labeled samples, and secondly, progressively fine-tuning using these samples.
arXiv Detail & Related papers (2023-08-14T06:04:50Z) - Losses over Labels: Weakly Supervised Learning via Direct Loss
Construction [71.11337906077483]
Programmable weak supervision is a growing paradigm within machine learning.
We propose Losses over Labels (LoL) as it creates losses directly from ofs without going through the intermediate step of a label.
We show that LoL improves upon existing weak supervision methods on several benchmark text and image classification tasks.
arXiv Detail & Related papers (2022-12-13T22:29:14Z) - Label Noise-Resistant Mean Teaching for Weakly Supervised Fake News
Detection [93.6222609806278]
We propose a novel label noise-resistant mean teaching approach (LNMT) for weakly supervised fake news detection.
LNMT leverages unlabeled news and feedback comments of users to enlarge the amount of training data.
LNMT establishes a mean teacher framework equipped with label propagation and label reliability estimation.
arXiv Detail & Related papers (2022-06-10T16:01:58Z) - Data Consistency for Weakly Supervised Learning [15.365232702938677]
Training machine learning models involves using large amounts of human-annotated data.
We propose a novel weak supervision algorithm that processes noisy labels, i.e., weak signals.
We show that it significantly outperforms state-of-the-art weak supervision methods on both text and image classification tasks.
arXiv Detail & Related papers (2022-02-08T16:48:19Z) - PseudoSeg: Designing Pseudo Labels for Semantic Segmentation [78.35515004654553]
We present a re-design of pseudo-labeling to generate structured pseudo labels for training with unlabeled or weakly-labeled data.
We demonstrate the effectiveness of the proposed pseudo-labeling strategy in both low-data and high-data regimes.
arXiv Detail & Related papers (2020-10-19T17:59:30Z) - Knowledge-Guided Multi-Label Few-Shot Learning for General Image
Recognition [75.44233392355711]
KGGR framework exploits prior knowledge of statistical label correlations with deep neural networks.
It first builds a structured knowledge graph to correlate different labels based on statistical label co-occurrence.
Then, it introduces the label semantics to guide learning semantic-specific features.
It exploits a graph propagation network to explore graph node interactions.
arXiv Detail & Related papers (2020-09-20T15:05:29Z) - Constrained Labeling for Weakly Supervised Learning [15.365232702938677]
We propose a simple data-free approach for combining weak supervision signals.
Our method is efficient and stable, converging after a few iterations of descent.
We show experimentally that our method outperforms other weak supervision methods on various text- and image-classification tasks.
arXiv Detail & Related papers (2020-09-15T21:30:53Z) - Foreseeing the Benefits of Incidental Supervision [83.08441990812636]
This paper studies whether we can, in a single framework, quantify the benefits of various types of incidental signals for a given target task without going through experiments.
We propose a unified PAC-Bayesian motivated informativeness measure, PABI, that characterizes the uncertainty reduction provided by incidental supervision signals.
arXiv Detail & Related papers (2020-06-09T20:59:42Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.