DAMP: Doubly Aligned Multilingual Parser for Task-Oriented Dialogue

• URL: http://arxiv.org/abs/2212.08054v2
• Date: Fri, 26 May 2023 05:48:06 GMT
• Title: DAMP: Doubly Aligned Multilingual Parser for Task-Oriented Dialogue
• Abstract summary: We show that constrastive alignment pretraining improves both English performance and transfer efficiency. Our Doubly Aligned Aligned Parsing (DAMP) improves mBERT transfer performance by 3x, 6x, and 81x on the Spanglish, Hinglish and Multilingual Task Oriented Parsing benchmarks respectively.
• Score: 44.30286950799403
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Modern virtual assistants use internal semantic parsing engines to convert user utterances to actionable commands. However, prior work has demonstrated that semantic parsing is a difficult multilingual transfer task with low transfer efficiency compared to other tasks. In global markets such as India and Latin America, this is a critical issue as switching between languages is prevalent for bilingual users. In this work we dramatically improve the zero-shot performance of a multilingual and codeswitched semantic parsing system using two stages of multilingual alignment. First, we show that constrastive alignment pretraining improves both English performance and transfer efficiency. We then introduce a constrained optimization approach for hyperparameter-free adversarial alignment during finetuning. Our Doubly Aligned Multilingual Parser (DAMP) improves mBERT transfer performance by 3x, 6x, and 81x on the Spanglish, Hinglish and Multilingual Task Oriented Parsing benchmarks respectively and outperforms XLM-R and mT5-Large using 3.2x fewer parameters.

Related papers

• GAIA-v2-LILT: Multilingual Adaptation of Agent Benchmark beyond Translation [2.845609289542755]
  Agent benchmarks remain largely English-centric, while their multilingual versions are often built with machine translation (MT) and limited post-editing.<n>We argue that, for agentic tasks, this minimal workflow can easily break benchmark validity through query-answer misalignment or culturally off-target context.<n>We propose a refined workflow for adapting English benchmarks into multiple languages with explicit functional alignment, cultural alignment, and difficulty calibration using both automated checks and human review.
  arXiv  Detail & Related papers  (2026-04-27T19:11:21Z)
• Language Fusion for Parameter-Efficient Cross-lingual Transfer [21.96231169571248]
  Fusion forLanguage Representations (FLARE) is a novel method that enhances representation quality and downstream performance for languages other than English. FLARE integrates source and target language representations within low-rank (LoRA) adapters using lightweight linear transformations. A series of experiments across representative cross-lingual natural language understanding tasks, including natural language inference, question-answering and sentiment analysis, demonstrate FLARE's effectiveness.
  arXiv  Detail & Related papers  (2025-01-12T18:02:29Z)
• Breaking the Script Barrier in Multilingual Pre-Trained Language Models with Transliteration-Based Post-Training Alignment [50.27950279695363]
  The transfer performance is often hindered when a low-resource target language is written in a different script than the high-resource source language. Inspired by recent work that uses transliteration to address this problem, our paper proposes a transliteration-based post-pretraining alignment (PPA) method.
  arXiv  Detail & Related papers  (2024-06-28T08:59:24Z)
• Enhancing Cross-lingual Natural Language Inference by Soft Prompting with Multilingual Verbalizer [52.46740830977898]
  Cross-lingual natural language inference is a fundamental problem in cross-lingual language understanding. We propose a novel Soft prompt learning framework with the Multilingual Verbalizer (SoftMV) for XNLI.
  arXiv  Detail & Related papers  (2023-05-22T06:31:29Z)
• Language-Aware Multilingual Machine Translation with Self-Supervised Learning [13.250011906361273]
  Multilingual machine translation (MMT) benefits from cross-lingual transfer but is a challenging multitask optimization problem. Self-supervised learning approaches have shown promise by improving translation performance as complementary tasks to the MMT task. We propose a novel but simple SSL task, concurrent denoising, that co-trains with the MMT task by concurrently denoising monolingual data on both the encoder and decoder.
  arXiv  Detail & Related papers  (2023-02-10T01:34:24Z)
• Parameter-Efficient Neural Reranking for Cross-Lingual and Multilingual Retrieval [66.69799641522133]
  State-of-the-art neural (re)rankers are notoriously data hungry. Current approaches typically transfer rankers trained on English data to other languages and cross-lingual setups by means of multilingual encoders. We show that two parameter-efficient approaches to cross-lingual transfer, namely Sparse Fine-Tuning Masks (SFTMs) and Adapters, allow for a more lightweight and more effective zero-shot transfer.
  arXiv  Detail & Related papers  (2022-04-05T15:44:27Z)
• Syntax-augmented Multilingual BERT for Cross-lingual Transfer [37.99210035238424]
  This work shows that explicitly providing language syntax and training mBERT helps cross-lingual transfer. Experiment results show that syntax-augmented mBERT improves cross-lingual transfer on popular benchmarks.
  arXiv  Detail & Related papers  (2021-06-03T21:12:50Z)
• Are Multilingual Models Effective in Code-Switching? [57.78477547424949]
  We study the effectiveness of multilingual language models to understand their capability and adaptability to the mixed-language setting. Our findings suggest that pre-trained multilingual models do not necessarily guarantee high-quality representations on code-switching.
  arXiv  Detail & Related papers  (2021-03-24T16:20:02Z)
• VECO: Variable and Flexible Cross-lingual Pre-training for Language Understanding and Generation [77.82373082024934]
  We plug a cross-attention module into the Transformer encoder to explicitly build the interdependence between languages. It can effectively avoid the degeneration of predicting masked words only conditioned on the context in its own language. The proposed cross-lingual model delivers new state-of-the-art results on various cross-lingual understanding tasks of the XTREME benchmark.
  arXiv  Detail & Related papers  (2020-10-30T03:41:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.