Disambiguation of weak supervision with exponential convergence rates

• URL: http://arxiv.org/abs/2102.02789v1
• Date: Thu, 4 Feb 2021 18:14:32 GMT
• Title: Disambiguation of weak supervision with exponential convergence rates
• Authors: Vivien Cabannes, Francis Bach, Alessandro Rudi
• Abstract summary: In supervised learning, data are annotated with incomplete yet discriminative information. In this paper, we focus on partial labelling, an instance of weak supervision where, from a given input, we are given a set of potential targets. We propose an empirical disambiguation algorithm to recover full supervision from weak supervision.
• Score: 88.99819200562784
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine learning approached through supervised learning requires expensive annotation of data. This motivates weakly supervised learning, where data are annotated with incomplete yet discriminative information. In this paper, we focus on partial labelling, an instance of weak supervision where, from a given input, we are given a set of potential targets. We review a disambiguation principle to recover full supervision from weak supervision, and propose an empirical disambiguation algorithm. We prove exponential convergence rates of our algorithm under classical learnability assumptions, and we illustrate the usefulness of our method on practical examples.

Related papers

• Label-Agnostic Forgetting: A Supervision-Free Unlearning in Deep Models [7.742594744641462]
  Machine unlearning aims to remove information derived from forgotten data while preserving that of the remaining dataset in a well-trained model. We propose a supervision-free unlearning approach that operates without the need for labels during the unlearning process.
  arXiv  Detail & Related papers  (2024-03-31T00:29:00Z)
• Consciousness-Inspired Spatio-Temporal Abstractions for Better Generalization in Reinforcement Learning [83.41487567765871]
  Skipper is a model-based reinforcement learning framework. It automatically generalizes the task given into smaller, more manageable subtasks. It enables sparse decision-making and focused abstractions on the relevant parts of the environment.
  arXiv  Detail & Related papers  (2023-09-30T02:25:18Z)
• Rethinking Weak Supervision in Helping Contrastive Learning [19.5649824524209]
  We explore the mechanical differences between semi-supervised and noisy-labeled information in helping contrastive learning. Specifically, we investigate the most intuitive paradigm of jointly training supervised and unsupervised contrastive losses. We prove that semi-supervised labels improve the downstream error bound whereas noisy labels have limited effects under such a paradigm.
  arXiv  Detail & Related papers  (2023-06-07T05:18:27Z)
• Label Propagation with Weak Supervision [47.52032178837098]
  We introduce a novel analysis of the classical label propagation algorithm (LPA) (Zhu & Ghahramani, 2002) We provide an error bound that exploits both the local geometric properties of the underlying graph and the quality of the prior information. We demonstrate the ability of our approach on multiple benchmark weakly supervised classification tasks, showing improvements upon existing semi-supervised and weakly supervised methods.
  arXiv  Detail & Related papers  (2022-10-07T14:53:02Z)
• Unsupervised Learning of Debiased Representations with Pseudo-Attributes [85.5691102676175]
  We propose a simple but effective debiasing technique in an unsupervised manner. We perform clustering on the feature embedding space and identify pseudoattributes by taking advantage of the clustering results. We then employ a novel cluster-based reweighting scheme for learning debiased representation.
  arXiv  Detail & Related papers  (2021-08-06T05:20:46Z)
• Adversarial Examples for Unsupervised Machine Learning Models [71.81480647638529]
  Adrial examples causing evasive predictions are widely used to evaluate and improve the robustness of machine learning models. We propose a framework of generating adversarial examples for unsupervised models and demonstrate novel applications to data augmentation.
  arXiv  Detail & Related papers  (2021-03-02T17:47:58Z)
• A Sober Look at the Unsupervised Learning of Disentangled Representations and their Evaluation [63.042651834453544]
  We show that the unsupervised learning of disentangled representations is impossible without inductive biases on both the models and the data. We observe that while the different methods successfully enforce properties "encouraged" by the corresponding losses, well-disentangled models seemingly cannot be identified without supervision. Our results suggest that future work on disentanglement learning should be explicit about the role of inductive biases and (implicit) supervision.
  arXiv  Detail & Related papers  (2020-10-27T10:17:15Z)
• Functional Regularization for Representation Learning: A Unified Theoretical Perspective [27.93916012334704]
  Unsupervised and self-supervised learning approaches have become a crucial tool to learn representations for downstream prediction tasks. We present a unifying perspective where several such approaches can be viewed as imposing a regularization on the representation via a learnable function using unlabeled data. We propose a discriminative theoretical framework for analyzing the sample complexity of these approaches, which generalizes the framework of (Balcan and Blum, 2010) to allow learnable regularization functions.
  arXiv  Detail & Related papers  (2020-08-06T04:06:04Z)
• Learning Adversarially Robust Representations via Worst-Case Mutual Information Maximization [15.087280646796527]
  Training machine learning models that are robust against adversarial inputs poses seemingly insurmountable challenges. We develop a notion of representation vulnerability that captures the maximum change of mutual information between the input and output distributions. We propose an unsupervised learning method for obtaining intrinsically robust representations by maximizing the worst-case mutual information.
  arXiv  Detail & Related papers  (2020-02-26T21:20:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.