End-to-End Slot Alignment and Recognition for Cross-Lingual NLU

• URL: http://arxiv.org/abs/2004.14353v2
• Date: Thu, 8 Oct 2020 04:36:04 GMT
• Title: End-to-End Slot Alignment and Recognition for Cross-Lingual NLU
• Abstract summary: We propose a novel end-to-end model that learns to align and predict target slot labels jointly for cross-lingual transfer. We introduce MultiATIS++, a new multilingual NLU corpus that extends the Multilingual ATIS corpus to nine languages across four language families. Results show that our method outperforms a simple label projection method using fast-align on most languages, and achieves competitive performance to the more complex, state-of-the-art projection method with only half of the training time.
• Score: 9.399840807973545
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Natural language understanding (NLU) in the context of goal-oriented dialog systems typically includes intent classification and slot labeling tasks. Existing methods to expand an NLU system to new languages use machine translation with slot label projection from source to the translated utterances, and thus are sensitive to projection errors. In this work, we propose a novel end-to-end model that learns to align and predict target slot labels jointly for cross-lingual transfer. We introduce MultiATIS++, a new multilingual NLU corpus that extends the Multilingual ATIS corpus to nine languages across four language families, and evaluate our method using the corpus. Results show that our method outperforms a simple label projection method using fast-align on most languages, and achieves competitive performance to the more complex, state-of-the-art projection method with only half of the training time. We release our MultiATIS++ corpus to the community to continue future research on cross-lingual NLU.

Related papers

• Generating Concept Lexicalizations via Dictionary-Based Cross-Lingual Sense Projection [11.136711217530157]
  We generate senses by associating target-language lemmas with existing lexical concepts via semantic projection.<n>Our method projects English synsets onto aligned target-language tokens and assigns the corresponding lemmas to those synsets.<n>We evaluate the method on multiple languages, comparing it to prior methods, as well as dictionary-based and large language model baselines.
  arXiv  Detail & Related papers  (2026-04-15T20:27:26Z)
• What Language is This? Ask Your Tokenizer [32.28976119949841]
  Language Identification (LID) is an important component of many multilingual natural language processing pipelines.<n>We introduce UniLID, a simple and efficient LID method based on the UnigramLM tokenization algorithm.<n>Our formulation is data- and compute-efficient, supports incremental addition of new languages without retraining existing models.
  arXiv  Detail & Related papers  (2026-02-19T18:58:39Z)
• Middle-Layer Representation Alignment for Cross-Lingual Transfer in Fine-Tuned LLMs [20.756383171892608]
  Middle layers exhibit strongest potential for cross-lingual alignment. Experiments on slot filling, machine translation, and structured text generation show consistent improvements in cross-lingual transfer. We show that separately trained alignment modules can be merged with existing task-specific modules, improving cross-lingual capabilities without full re-training.
  arXiv  Detail & Related papers  (2025-02-20T18:45:43Z)
• UniBridge: A Unified Approach to Cross-Lingual Transfer Learning for Low-Resource Languages [2.4510449184543925]
  We introduce UniBridge, a comprehensive approach to improve the effectiveness of Cross-Lingual Transfer Learning. Our approach tackles two essential elements of a language model: the initialization of embeddings and the optimal vocabulary size. Our experiments across multilingual datasets show that our approach greatly improves the F1-Score in several languages.
  arXiv  Detail & Related papers  (2024-06-14T04:55:30Z)
• Efficient Spoken Language Recognition via Multilabel Classification [53.662747523872305]
  We show that our models obtain competitive results while being orders of magnitude smaller and faster than current state-of-the-art methods. Our multilabel strategy is more robust to unseen non-target languages compared to multiclass classification.
  arXiv  Detail & Related papers  (2023-06-02T23:04:19Z)
• Multi-level Contrastive Learning for Cross-lingual Spoken Language Understanding [90.87454350016121]
  We develop novel code-switching schemes to generate hard negative examples for contrastive learning at all levels. We develop a label-aware joint model to leverage label semantics for cross-lingual knowledge transfer.
  arXiv  Detail & Related papers  (2022-05-07T13:44:28Z)
• Exposing Cross-Lingual Lexical Knowledge from Multilingual Sentence Encoders [85.80950708769923]
  We probe multilingual language models for the amount of cross-lingual lexical knowledge stored in their parameters, and compare them against the original multilingual LMs. We also devise a novel method to expose this knowledge by additionally fine-tuning multilingual models. We report substantial gains on standard benchmarks.
  arXiv  Detail & Related papers  (2022-04-30T13:23:16Z)
• Por Qu\'e N\~ao Utiliser Alla Spr{\aa}k? Mixed Training with Gradient Optimization in Few-Shot Cross-Lingual Transfer [2.7213511121305465]
  We propose a one-step mixed training method that trains on both source and target data. We use one model to handle all target languages simultaneously to avoid excessively language-specific models. Our proposed method achieves state-of-the-art performance on all tasks and outperforms target-adapting by a large margin.
  arXiv  Detail & Related papers  (2022-04-29T04:05:02Z)
• Learning Contextualised Cross-lingual Word Embeddings and Alignments for Extremely Low-Resource Languages Using Parallel Corpora [63.5286019659504]
  We propose a new approach for learning contextualised cross-lingual word embeddings based on a small parallel corpus. Our method obtains word embeddings via an LSTM encoder-decoder model that simultaneously translates and reconstructs an input sentence.
  arXiv  Detail & Related papers  (2020-10-27T22:24:01Z)
• FILTER: An Enhanced Fusion Method for Cross-lingual Language Understanding [85.29270319872597]
  We propose an enhanced fusion method that takes cross-lingual data as input for XLM finetuning. During inference, the model makes predictions based on the text input in the target language and its translation in the source language. To tackle this issue, we propose an additional KL-divergence self-teaching loss for model training, based on auto-generated soft pseudo-labels for translated text in the target language.
  arXiv  Detail & Related papers  (2020-09-10T22:42:15Z)
• Robust Cross-lingual Embeddings from Parallel Sentences [65.85468628136927]
  We propose a bilingual extension of the CBOW method which leverages sentence-aligned corpora to obtain robust cross-lingual word representations. Our approach significantly improves crosslingual sentence retrieval performance over all other approaches. It also achieves parity with a deep RNN method on a zero-shot cross-lingual document classification task.
  arXiv  Detail & Related papers  (2019-12-28T16:18:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.