HIT-SCIR at MMNLU-22: Consistency Regularization for Multilingual Spoken Language Understanding

• URL: http://arxiv.org/abs/2301.02010v1
• Date: Thu, 5 Jan 2023 11:21:15 GMT
• Title: HIT-SCIR at MMNLU-22: Consistency Regularization for Multilingual Spoken Language Understanding
• Authors: Bo Zheng, Zhouyang Li, Fuxuan Wei, Qiguang Chen, Libo Qin, Wanxiang Che
• Abstract summary: We propose to use consistency regularization based on a hybrid data augmentation strategy. We conduct experiments on the MASSIVE dataset under both full-dataset and zero-shot settings. Our proposed method improves the performance on both intent detection and slot filling tasks.
• Score: 56.756090143062536
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multilingual spoken language understanding (SLU) consists of two sub-tasks, namely intent detection and slot filling. To improve the performance of these two sub-tasks, we propose to use consistency regularization based on a hybrid data augmentation strategy. The consistency regularization enforces the predicted distributions for an example and its semantically equivalent augmentation to be consistent. We conduct experiments on the MASSIVE dataset under both full-dataset and zero-shot settings. Experimental results demonstrate that our proposed method improves the performance on both intent detection and slot filling tasks. Our system\footnote{The code will be available at \url{https://github.com/bozheng-hit/MMNLU-22-HIT-SCIR}.} ranked 1st in the MMNLU-22 competition under the full-dataset setting.

Related papers

• Towards Spoken Language Understanding via Multi-level Multi-grained Contrastive Learning [50.1035273069458]
  Spoken language understanding (SLU) is a core task in task-oriented dialogue systems. We propose a multi-level MMCL framework to apply contrastive learning at three levels, including utterance level, slot level, and word level. Our framework achieves new state-of-the-art results on two public multi-intent SLU datasets.
  arXiv  Detail & Related papers  (2024-05-31T14:34:23Z)
• Unify word-level and span-level tasks: NJUNLP's Participation for the WMT2023 Quality Estimation Shared Task [59.46906545506715]
  We introduce the NJUNLP team to the WMT 2023 Quality Estimation (QE) shared task. Our team submitted predictions for the English-German language pair on all two sub-tasks. Our models achieved the best results in English-German for both word-level and fine-grained error span detection sub-tasks.
  arXiv  Detail & Related papers  (2023-09-23T01:52:14Z)
• Bridging Cross-Lingual Gaps During Leveraging the Multilingual Sequence-to-Sequence Pretraining for Text Generation [80.16548523140025]
  We extend the vanilla pretrain-finetune pipeline with extra code-switching restore task to bridge the gap between the pretrain and finetune stages. Our approach could narrow the cross-lingual sentence representation distance and improve low-frequency word translation with trivial computational cost.
  arXiv  Detail & Related papers  (2022-04-16T16:08:38Z)
• A Joint and Domain-Adaptive Approach to Spoken Language Understanding [30.164751046395573]
  Spoken Language Understanding (SLU) is composed of two subtasks: intent detection (ID) and slot filling (SF) One jointly tackles these two subtasks to improve their prediction accuracy, and the other focuses on the domain-adaptation ability of one of the subtasks. In this paper, we propose a joint and domain adaptive approach to SLU.
  arXiv  Detail & Related papers  (2021-07-25T09:38:42Z)
• Consistency Regularization for Cross-Lingual Fine-Tuning [61.08704789561351]
  We propose to improve cross-lingual fine-tuning with consistency regularization. Specifically, we use example consistency regularization to penalize the prediction sensitivity to four types of data augmentations. Experimental results on the XTREME benchmark show that our method significantly improves cross-lingual fine-tuning across various tasks.
  arXiv  Detail & Related papers  (2021-06-15T15:35:44Z)
• MCL@IITK at SemEval-2021 Task 2: Multilingual and Cross-lingual Word-in-Context Disambiguation using Augmented Data, Signals, and Transformers [1.869621561196521]
  We present our approach for solving the SemEval 2021 Task 2: Multilingual and Cross-lingual Word-in-Context Disambiguation (MCL-WiC) The goal is to detect whether a given word common to both the sentences evokes the same meaning. We submit systems for both the settings - Multilingual and Cross-Lingual.
  arXiv  Detail & Related papers  (2021-04-04T08:49:28Z)
• Explicit Alignment Objectives for Multilingual Bidirectional Encoders [111.65322283420805]
  We present a new method for learning multilingual encoders, AMBER (Aligned Multilingual Bi-directional EncodeR) AMBER is trained on additional parallel data using two explicit alignment objectives that align the multilingual representations at different granularities. Experimental results show that AMBER obtains gains of up to 1.1 average F1 score on sequence tagging and up to 27.3 average accuracy on retrieval over the XLMR-large model.
  arXiv  Detail & Related papers  (2020-10-15T18:34:13Z)
• Dual Learning for Semi-Supervised Natural Language Understanding [29.692288627633374]
  Natural language understanding (NLU) converts sentences into structured semantic forms. We introduce a dual task of NLU, semantic-to-sentence generation (SSG) We propose a new framework for semi-supervised NLU with the corresponding dual model.
  arXiv  Detail & Related papers  (2020-04-26T07:17:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.