Foreseeing the Benefits of Incidental Supervision

• URL: http://arxiv.org/abs/2006.05500v2
• Date: Fri, 10 Sep 2021 16:32:23 GMT
• Title: Foreseeing the Benefits of Incidental Supervision
• Authors: Hangfeng He, Mingyuan Zhang, Qiang Ning, Dan Roth
• Abstract summary: This paper studies whether we can, in a single framework, quantify the benefits of various types of incidental signals for a given target task without going through experiments. We propose a unified PAC-Bayesian motivated informativeness measure, PABI, that characterizes the uncertainty reduction provided by incidental supervision signals.
• Score: 83.08441990812636
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Real-world applications often require improved models by leveraging a range of cheap incidental supervision signals. These could include partial labels, noisy labels, knowledge-based constraints, and cross-domain or cross-task annotations -- all having statistical associations with gold annotations but not exactly the same. However, we currently lack a principled way to measure the benefits of these signals to a given target task, and the common practice of evaluating these benefits is through exhaustive experiments with various models and hyperparameters. This paper studies whether we can, in a single framework, quantify the benefits of various types of incidental signals for a given target task without going through combinatorial experiments. We propose a unified PAC-Bayesian motivated informativeness measure, PABI, that characterizes the uncertainty reduction provided by incidental supervision signals. We demonstrate PABI's effectiveness by quantifying the value added by various types of incidental signals to sequence tagging tasks. Experiments on named entity recognition (NER) and question answering (QA) show that PABI's predictions correlate well with learning performance, providing a promising way to determine, ahead of learning, which supervision signals would be beneficial.

Related papers

• Learning Contrastive Feature Representations for Facial Action Unit Detection [13.834540490373818]
  Facial action unit (AU) detection has long encountered the challenge of detecting subtle feature differences when AUs activate. We introduce a novel contrastive learning framework aimed for AU detection that incorporates both self-supervised and supervised signals.
  arXiv  Detail & Related papers  (2024-02-09T03:48:20Z)
• Mitigating Shortcut Learning with Diffusion Counterfactuals and Diverse Ensembles [95.49699178874683]
  We propose DiffDiv, an ensemble diversification framework exploiting Diffusion Probabilistic Models (DPMs) We show that DPMs can generate images with novel feature combinations, even when trained on samples displaying correlated input features. We show that DPM-guided diversification is sufficient to remove dependence on shortcut cues, without a need for additional supervised signals.
  arXiv  Detail & Related papers  (2023-11-23T15:47:33Z)
• Leveraging Diffusion Disentangled Representations to Mitigate Shortcuts in Underspecified Visual Tasks [92.32670915472099]
  We propose an ensemble diversification framework exploiting the generation of synthetic counterfactuals using Diffusion Probabilistic Models (DPMs) We show that diffusion-guided diversification can lead models to avert attention from shortcut cues, achieving ensemble diversity performance comparable to previous methods requiring additional data collection.
  arXiv  Detail & Related papers  (2023-10-03T17:37:52Z)
• Unsupervised Visible-Infrared Person ReID by Collaborative Learning with Neighbor-Guided Label Refinement [53.044703127757295]
  Unsupervised learning visible-infrared person re-identification (USL-VI-ReID) aims at learning modality-invariant features from unlabeled cross-modality dataset. We propose a Dual Optimal Transport Label Assignment (DOTLA) framework to simultaneously assign the generated labels from one modality to its counterpart modality. The proposed DOTLA mechanism formulates a mutual reinforcement and efficient solution to cross-modality data association, which could effectively reduce the side-effects of some insufficient and noisy label associations.
  arXiv  Detail & Related papers  (2023-05-22T04:40:30Z)
• Be Your Own Neighborhood: Detecting Adversarial Example by the Neighborhood Relations Built on Self-Supervised Learning [64.78972193105443]
  This paper presents a novel AE detection framework, named trustworthy for predictions. performs the detection by distinguishing the AE's abnormal relation with its augmented versions. An off-the-shelf Self-Supervised Learning (SSL) model is used to extract the representation and predict the label.
  arXiv  Detail & Related papers  (2022-08-31T08:18:44Z)
• XPASC: Measuring Generalization in Weak Supervision [5.197057543520865]
  We introduce XPASC (eXPlainability-Association SCore) for measuring the generalization of a model trained with a weakly supervised dataset. We use XPASC to analyze KnowMAN, an adversarial architecture intended to control the degree of generalization from the labeling functions.
  arXiv  Detail & Related papers  (2022-06-03T08:16:50Z)
• Feature Diversity Learning with Sample Dropout for Unsupervised Domain Adaptive Person Re-identification [0.0]
  This paper proposes a new approach to learn the feature representation with better generalization ability through limiting noisy pseudo labels. We put forward a brand-new method referred as to Feature Diversity Learning (FDL) under the classic mutual-teaching architecture. Experimental results show that our proposed FDL-SD achieves the state-of-the-art performance on multiple benchmark datasets.
  arXiv  Detail & Related papers  (2022-01-25T10:10:48Z)
• WSSOD: A New Pipeline for Weakly- and Semi-Supervised Object Detection [75.80075054706079]
  We propose a weakly- and semi-supervised object detection framework (WSSOD) An agent detector is first trained on a joint dataset and then used to predict pseudo bounding boxes on weakly-annotated images. The proposed framework demonstrates remarkable performance on PASCAL-VOC and MSCOCO benchmark, achieving a high performance comparable to those obtained in fully-supervised settings.
  arXiv  Detail & Related papers  (2021-05-21T11:58:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.