Toward Open-domain Slot Filling via Self-supervised Co-training

• URL: http://arxiv.org/abs/2303.13801v1
• Date: Fri, 24 Mar 2023 04:51:22 GMT
• Title: Toward Open-domain Slot Filling via Self-supervised Co-training
• Authors: Adib Mosharrof, Moghis Fereidouni, A.B. Siddique
• Abstract summary: Slot filling is one of the critical tasks in modern conversational systems. We propose a Self-supervised Co-training framework, called SCot, that requires zero in-domain manually labeled training examples. Our evaluations show that SCot outperforms state-of-the-art models by 45.57% and 37.56% on SGD and MultiWoZ datasets.
• Score: 2.7178968279054936
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Slot filling is one of the critical tasks in modern conversational systems. The majority of existing literature employs supervised learning methods, which require labeled training data for each new domain. Zero-shot learning and weak supervision approaches, among others, have shown promise as alternatives to manual labeling. Nonetheless, these learning paradigms are significantly inferior to supervised learning approaches in terms of performance. To minimize this performance gap and demonstrate the possibility of open-domain slot filling, we propose a Self-supervised Co-training framework, called SCot, that requires zero in-domain manually labeled training examples and works in three phases. Phase one acquires two sets of complementary pseudo labels automatically. Phase two leverages the power of the pre-trained language model BERT, by adapting it for the slot filling task using these sets of pseudo labels. In phase three, we introduce a self-supervised cotraining mechanism, where both models automatically select highconfidence soft labels to further improve the performance of the other in an iterative fashion. Our thorough evaluations show that SCot outperforms state-of-the-art models by 45.57% and 37.56% on SGD and MultiWoZ datasets, respectively. Moreover, our proposed framework SCot achieves comparable performance when compared to state-of-the-art fully supervised models.

Related papers

• Unified Speech Recognition: A Single Model for Auditory, Visual, and Audiovisual Inputs [73.74375912785689]
  This paper proposes unified training strategies for speech recognition systems. We demonstrate that training a single model for all three tasks enhances VSR and AVSR performance. We also introduce a greedy pseudo-labelling approach to more effectively leverage unlabelled samples.
  arXiv  Detail & Related papers  (2024-11-04T16:46:53Z)
• Meta Co-Training: Two Views are Better than One [4.050257210426548]
  We present Meta Co-Training which is an extension of the successful Meta Pseudo Labels approach to two views. Our method achieves new state-of-the-art performance on ImageNet-10% with very few training resources.
  arXiv  Detail & Related papers  (2023-11-29T21:11:58Z)
• One-bit Supervision for Image Classification: Problem, Solution, and Beyond [114.95815360508395]
  This paper presents one-bit supervision, a novel setting of learning with fewer labels, for image classification. We propose a multi-stage training paradigm and incorporate negative label suppression into an off-the-shelf semi-supervised learning algorithm. In multiple benchmarks, the learning efficiency of the proposed approach surpasses that using full-bit, semi-supervised supervision.
  arXiv  Detail & Related papers  (2023-11-26T07:39:00Z)
• Co-guiding for Multi-intent Spoken Language Understanding [53.30511968323911]
  We propose a novel model termed Co-guiding Net, which implements a two-stage framework achieving the mutual guidances between the two tasks. For the first stage, we propose single-task supervised contrastive learning, and for the second stage, we propose co-guiding supervised contrastive learning. Experiment results on multi-intent SLU show that our model outperforms existing models by a large margin.
  arXiv  Detail & Related papers  (2023-11-22T08:06:22Z)
• Unsupervised 3D registration through optimization-guided cyclical self-training [71.75057371518093]
  State-of-the-art deep learning-based registration methods employ three different learning strategies. We propose a novel self-supervised learning paradigm for unsupervised registration, relying on self-training. We evaluate the method for abdomen and lung registration, consistently surpassing metric-based supervision and outperforming diverse state-of-the-art competitors.
  arXiv  Detail & Related papers  (2023-06-29T14:54:10Z)
• Integrated Weak Learning [25.47289093245517]
  Integrated Weak Learning is a principled framework that integrates weak supervision into the training process of machine learning models. We show that our approach outperforms existing weak learning techniques across a set of 6 benchmark classification datasets.
  arXiv  Detail & Related papers  (2022-06-19T22:13:59Z)
• Active Self-Semi-Supervised Learning for Few Labeled Samples [4.713652957384158]
  Training deep models with limited annotations poses a significant challenge when applied to diverse practical domains. We propose a simple yet effective framework, active self-semi-supervised learning (AS3L) AS3L bootstraps semi-supervised models with prior pseudo-labels (PPL) We develop active learning and label propagation strategies to obtain accurate PPL.
  arXiv  Detail & Related papers  (2022-03-09T07:45:05Z)
• Self-Supervised Models are Continual Learners [79.70541692930108]
  We show that self-supervised loss functions can be seamlessly converted into distillation mechanisms for Continual Learning. We devise a framework for Continual self-supervised visual representation Learning that significantly improves the quality of the learned representations.
  arXiv  Detail & Related papers  (2021-12-08T10:39:13Z)
• Few-Shot Named Entity Recognition: A Comprehensive Study [92.40991050806544]
  We investigate three schemes to improve the model generalization ability for few-shot settings. We perform empirical comparisons on 10 public NER datasets with various proportions of labeled data. We create new state-of-the-art results on both few-shot and training-free settings.
  arXiv  Detail & Related papers  (2020-12-29T23:43:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.