HierarchicalContrast: A Coarse-to-Fine Contrastive Learning Framework for Cross-Domain Zero-Shot Slot Filling

• URL: http://arxiv.org/abs/2310.09135v2
• Date: Fri, 20 Oct 2023 11:29:53 GMT
• Title: HierarchicalContrast: A Coarse-to-Fine Contrastive Learning Framework for Cross-Domain Zero-Shot Slot Filling
• Authors: Junwen Zhang and Yin Zhang
• Abstract summary: Cross-domain zero-shot slot filling plays a vital role in leveraging source domain knowledge to learn a model. Existing state-of-the-art zero-shot slot filling methods have limited generalization ability in target domain. We present a novel Hierarchical Contrastive Learning Framework (HiCL) for zero-shot slot filling.
• Score: 4.1940152307593515
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In task-oriented dialogue scenarios, cross-domain zero-shot slot filling plays a vital role in leveraging source domain knowledge to learn a model with high generalization ability in unknown target domain where annotated data is unavailable. However, the existing state-of-the-art zero-shot slot filling methods have limited generalization ability in target domain, they only show effective knowledge transfer on seen slots and perform poorly on unseen slots. To alleviate this issue, we present a novel Hierarchical Contrastive Learning Framework (HiCL) for zero-shot slot filling. Specifically, we propose a coarse- to fine-grained contrastive learning based on Gaussian-distributed embedding to learn the generalized deep semantic relations between utterance-tokens, by optimizing inter- and intra-token distribution distance. This encourages HiCL to generalize to the slot types unseen at training phase. Furthermore, we present a new iterative label set semantics inference method to unbiasedly and separately evaluate the performance of unseen slot types which entangled with their counterparts (i.e., seen slot types) in the previous zero-shot slot filling evaluation methods. The extensive empirical experiments on four datasets demonstrate that the proposed method achieves comparable or even better performance than the current state-of-the-art zero-shot slot filling approaches.

Related papers

• Adaptive End-to-End Metric Learning for Zero-Shot Cross-Domain Slot Filling [2.6056468338837457]
  Slot filling poses a critical challenge to handle a novel domain whose samples are never seen during training. Most prior works deal with this problem in a two-pass pipeline manner based on metric learning. We propose a new adaptive end-to-end metric learning scheme for the challenging zero-shot slot filling.
  arXiv  Detail & Related papers  (2023-10-23T19:01:16Z)
• Slot Induction via Pre-trained Language Model Probing and Multi-level Contrastive Learning [62.839109775887025]
  Slot Induction (SI) task whose objective is to induce slot boundaries without explicit knowledge of token-level slot annotations. We propose leveraging Unsupervised Pre-trained Language Model (PLM) Probing and Contrastive Learning mechanism to exploit unsupervised semantic knowledge extracted from PLM. Our approach is shown to be effective in SI task and capable of bridging the gaps with token-level supervised models on two NLU benchmark datasets.
  arXiv  Detail & Related papers  (2023-08-09T05:08:57Z)
• Generative Zero-Shot Prompt Learning for Cross-Domain Slot Filling with Inverse Prompting [27.186526104248696]
  Cross-domain slot filling aims to transfer knowledge from the labeled domain to the unlabeled target domain. We propose a generative zero-shot prompt learning framework for cross-domain slot filling. Experiments and analysis demonstrate the effectiveness of our proposed framework.
  arXiv  Detail & Related papers  (2023-07-06T07:53:46Z)
• Robust Saliency-Aware Distillation for Few-shot Fine-grained Visual Recognition [57.08108545219043]
  Recognizing novel sub-categories with scarce samples is an essential and challenging research topic in computer vision. Existing literature addresses this challenge by employing local-based representation approaches. This article proposes a novel model, Robust Saliency-aware Distillation (RSaD), for few-shot fine-grained visual recognition.
  arXiv  Detail & Related papers  (2023-05-12T00:13:17Z)
• Few-Shot Point Cloud Semantic Segmentation via Contrastive Self-Supervision and Multi-Resolution Attention [6.350163959194903]
  We propose a contrastive self-supervision framework for few-shot learning pretrain. Specifically, we implement a novel contrastive learning approach with a learnable augmentor for a 3D point cloud. We develop a multi-resolution attention module using both the nearest and farthest points to extract the local and global point information more effectively.
  arXiv  Detail & Related papers  (2023-02-21T07:59:31Z)
• Bridge to Target Domain by Prototypical Contrastive Learning and Label Confusion: Re-explore Zero-Shot Learning for Slot Filling [18.19818129121059]
  Cross-domain slot filling alleviates the data dependence in the case of data scarcity in the target domain. We propose a novel approach based on prototypical contrastive learning with a dynamic label confusion strategy for zero-shot slot filling. Our model achieves significant improvement on the unseen slots, while also set new state-of-the-arts on slot filling task.
  arXiv  Detail & Related papers  (2021-10-07T15:50:56Z)
• Leveraging Slot Descriptions for Zero-Shot Cross-Domain Dialogue State Tracking [50.04597636485369]
  Zero-shot cross-domain dialogue state tracking (DST) enables us to handle task-oriented dialogue in unseen domains without the expense of collecting in-domain data. We propose a slot description enhanced generative approach for zero-shot cross-domain DST.
  arXiv  Detail & Related papers  (2021-05-10T09:34:01Z)
• Learning Invariant Representations and Risks for Semi-supervised Domain Adaptation [109.73983088432364]
  We propose the first method that aims to simultaneously learn invariant representations and risks under the setting of semi-supervised domain adaptation (Semi-DA) We introduce the LIRR algorithm for jointly textbfLearning textbfInvariant textbfRepresentations and textbfRisks.
  arXiv  Detail & Related papers  (2020-10-09T15:42:35Z)
• Coach: A Coarse-to-Fine Approach for Cross-domain Slot Filling [65.09621991654745]
  Cross-domain slot filling is an essential task in task-oriented dialog systems. We propose a Coarse-to-fine approach (Coach) for cross-domain slot filling. Experimental results show that our model significantly outperforms state-of-the-art approaches in slot filling.
  arXiv  Detail & Related papers  (2020-04-24T13:07:12Z)
• Generalized Zero-Shot Learning Via Over-Complete Distribution [79.5140590952889]
  We propose to generate an Over-Complete Distribution (OCD) using Conditional Variational Autoencoder (CVAE) of both seen and unseen classes. The effectiveness of the framework is evaluated using both Zero-Shot Learning and Generalized Zero-Shot Learning protocols.
  arXiv  Detail & Related papers  (2020-04-01T19:05:28Z)
• From Anchor Generation to Distribution Alignment: Learning a Discriminative Embedding Space for Zero-Shot Recognition [46.47620562161315]
  In zero-shot learning (ZSL), the samples to be classified are usually projected into side information templates such as attributes. We propose a novel framework called Discriminative Anchor Generation and Distribution Alignment Model (DAGDA) Firstly, in order to rectify the distribution of original templates, a diffusion based graph convolutional network, which can explicitly model the interaction between class and side information, is proposed to produce discriminative anchors. Secondly, to further align the samples with the corresponding anchors in anchor space, which aims to refine the distribution in a fine-grained manner, we introduce a semantic relation regularization
  arXiv  Detail & Related papers  (2020-02-10T05:25:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.