Generative Zero-Shot Prompt Learning for Cross-Domain Slot Filling with Inverse Prompting

• URL: http://arxiv.org/abs/2307.02830v1
• Date: Thu, 6 Jul 2023 07:53:46 GMT
• Title: Generative Zero-Shot Prompt Learning for Cross-Domain Slot Filling with Inverse Prompting
• Authors: Xuefeng Li, Liwen Wang, Guanting Dong, Keqing He, Jinzheng Zhao, Hao Lei, Jiachi Liu, Weiran Xu
• Abstract summary: Cross-domain slot filling aims to transfer knowledge from the labeled domain to the unlabeled target domain. We propose a generative zero-shot prompt learning framework for cross-domain slot filling. Experiments and analysis demonstrate the effectiveness of our proposed framework.
• Score: 27.186526104248696
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Zero-shot cross-domain slot filling aims to transfer knowledge from the labeled source domain to the unlabeled target domain. Existing models either encode slot descriptions and examples or design handcrafted question templates using heuristic rules, suffering from poor generalization capability or robustness. In this paper, we propose a generative zero-shot prompt learning framework for cross-domain slot filling, both improving generalization and robustness than previous work. Besides, we introduce a novel inverse prompting strategy to distinguish different slot types to avoid the multiple prediction problem, and an efficient prompt-tuning strategy to boost higher performance by only training fewer prompt parameters. Experiments and analysis demonstrate the effectiveness of our proposed framework, especially huge improvements (+13.44% F1) on the unseen slots.

Related papers

• Memory-Efficient Prompt Tuning for Incremental Histopathology Classification [69.46798702300042]
  We present a memory-efficient prompt tuning framework to cultivate model generalization potential in economical memory cost. We have extensively evaluated our framework with two histopathology tasks, i.e., breast cancer metastasis classification and epithelium-stroma tissue classification.
  arXiv  Detail & Related papers  (2024-01-22T03:24:45Z)
• HCVP: Leveraging Hierarchical Contrastive Visual Prompt for Domain Generalization [69.33162366130887]
  Domain Generalization (DG) endeavors to create machine learning models that excel in unseen scenarios by learning invariant features. We introduce a novel method designed to supplement the model with domain-level and task-specific characteristics. This approach aims to guide the model in more effectively separating invariant features from specific characteristics, thereby boosting the generalization.
  arXiv  Detail & Related papers  (2024-01-18T04:23:21Z)
• TACIT: A Target-Agnostic Feature Disentanglement Framework for Cross-Domain Text Classification [17.19214732926589]
  Cross-domain text classification aims to transfer models from label-rich source domains to label-poor target domains. This paper proposes TACIT, a target domain feature disentanglement framework which adaptively decouples robust and unrobust features. Our framework achieves comparable results to state-of-the-art baselines while utilizing only source domain data.
  arXiv  Detail & Related papers  (2023-12-25T02:52:36Z)
• HierarchicalContrast: A Coarse-to-Fine Contrastive Learning Framework for Cross-Domain Zero-Shot Slot Filling [4.1940152307593515]
  Cross-domain zero-shot slot filling plays a vital role in leveraging source domain knowledge to learn a model. Existing state-of-the-art zero-shot slot filling methods have limited generalization ability in target domain. We present a novel Hierarchical Contrastive Learning Framework (HiCL) for zero-shot slot filling.
  arXiv  Detail & Related papers  (2023-10-13T14:23:33Z)
• Activate and Reject: Towards Safe Domain Generalization under Category Shift [71.95548187205736]
  We study a practical problem of Domain Generalization under Category Shift (DGCS) It aims to simultaneously detect unknown-class samples and classify known-class samples in the target domains. Compared to prior DG works, we face two new challenges: 1) how to learn the concept of unknown'' during training with only source known-class samples, and 2) how to adapt the source-trained model to unseen environments.
  arXiv  Detail & Related papers  (2023-10-07T07:53:12Z)
• Prompting Diffusion Representations for Cross-Domain Semantic Segmentation [101.04326113360342]
  diffusion-pretraining achieves extraordinary domain generalization results for semantic segmentation. We introduce a scene prompt and a prompt randomization strategy to help further disentangle the domain-invariant information when training the segmentation head.
  arXiv  Detail & Related papers  (2023-07-05T09:28:25Z)
• Randomized Adversarial Style Perturbations for Domain Generalization [49.888364462991234]
  We propose a novel domain generalization technique, referred to as Randomized Adversarial Style Perturbation (RASP) The proposed algorithm perturbs the style of a feature in an adversarial direction towards a randomly selected class, and makes the model learn against being misled by the unexpected styles observed in unseen target domains. We evaluate the proposed algorithm via extensive experiments on various benchmarks and show that our approach improves domain generalization performance, especially in large-scale benchmarks.
  arXiv  Detail & Related papers  (2023-04-04T17:07:06Z)
• Learning Domain Invariant Prompt for Vision-Language Models [31.581652862478965]
  We propose a novel prompt learning paradigm that directly generates emphdomain invariant prompt that can be generalized to unseen domains, called MetaPrompt. Our method consistently and significantly outperforms existing methods.
  arXiv  Detail & Related papers  (2022-12-08T11:23:24Z)
• Bridge to Target Domain by Prototypical Contrastive Learning and Label Confusion: Re-explore Zero-Shot Learning for Slot Filling [18.19818129121059]
  Cross-domain slot filling alleviates the data dependence in the case of data scarcity in the target domain. We propose a novel approach based on prototypical contrastive learning with a dynamic label confusion strategy for zero-shot slot filling. Our model achieves significant improvement on the unseen slots, while also set new state-of-the-arts on slot filling task.
  arXiv  Detail & Related papers  (2021-10-07T15:50:56Z)
• Coach: A Coarse-to-Fine Approach for Cross-domain Slot Filling [65.09621991654745]
  Cross-domain slot filling is an essential task in task-oriented dialog systems. We propose a Coarse-to-fine approach (Coach) for cross-domain slot filling. Experimental results show that our model significantly outperforms state-of-the-art approaches in slot filling.
  arXiv  Detail & Related papers  (2020-04-24T13:07:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.