LaDA: Latent Dialogue Action For Zero-shot Cross-lingual Neural Network Language Modeling

• URL: http://arxiv.org/abs/2308.02903v1
• Date: Sat, 5 Aug 2023 15:51:45 GMT
• Title: LaDA: Latent Dialogue Action For Zero-shot Cross-lingual Neural Network Language Modeling
• Authors: Zhanyu Ma and Jian Ye and Shuang Cheng
• Abstract summary: Cross-lingual adaptation has proven effective in spoken language understanding systems with limited resources. Existing methods are frequently unsatisfactory for intent detection and slot filling. Latent Dialogue Action layer is proposed to optimize decoding strategy.
• Score: 20.002861239367704
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Cross-lingual adaptation has proven effective in spoken language understanding (SLU) systems with limited resources. Existing methods are frequently unsatisfactory for intent detection and slot filling, particularly for distant languages that differ significantly from the source language in scripts, morphology, and syntax. Latent Dialogue Action (LaDA) layer is proposed to optimize decoding strategy in order to address the aforementioned issues. The model consists of an additional layer of latent dialogue action. It enables our model to improve a system's capability of handling conversations with complex multilingual intent and slot values of distant languages. To the best of our knowledge, this is the first exhaustive investigation of the use of latent variables for optimizing cross-lingual SLU policy during the decode stage. LaDA obtains state-of-the-art results on public datasets for both zero-shot and few-shot adaptation.

Related papers

• Lens: Rethinking Multilingual Enhancement for Large Language Models [70.85065197789639]
  Lens is a novel approach to enhance multilingual capabilities of large language models (LLMs) It operates by manipulating the hidden representations within the language-agnostic and language-specific subspaces from top layers of LLMs. It achieves superior results with much fewer computational resources compared to existing post-training approaches.
  arXiv  Detail & Related papers  (2024-10-06T08:51:30Z)
• ChatZero:Zero-shot Cross-Lingual Dialogue Generation via Pseudo-Target Language [53.8622516025736]
  We propose a novel end-to-end zero-shot dialogue generation model ChatZero based on cross-lingual code-switching method. Experiments on the multilingual DailyDialog and DSTC7-AVSD datasets demonstrate that ChatZero can achieve more than 90% of the original performance.
  arXiv  Detail & Related papers  (2024-08-16T13:11:53Z)
• Language Portability Strategies for Open-domain Dialogue with Pre-trained Language Models from High to Low Resource Languages [1.7436854281619139]
  We propose a study of linguistic portability strategies of large pre-trained language models (PLMs) used for open-domain dialogue systems. In particular the target low-resource language (L_T) will be simulated with French, as it lacks of task-specific resources.
  arXiv  Detail & Related papers  (2024-07-01T14:20:54Z)
• Efficiently Aligned Cross-Lingual Transfer Learning for Conversational Tasks using Prompt-Tuning [98.60739735409243]
  Cross-lingual transfer of language models trained on high-resource languages like English has been widely studied for many NLP tasks. We introduce XSGD for cross-lingual alignment pretraining, a parallel and large-scale multilingual conversation dataset. To facilitate aligned cross-lingual representations, we develop an efficient prompt-tuning-based method for learning alignment prompts.
  arXiv  Detail & Related papers  (2023-04-03T18:46:01Z)
• Multi-level Contrastive Learning for Cross-lingual Spoken Language Understanding [90.87454350016121]
  We develop novel code-switching schemes to generate hard negative examples for contrastive learning at all levels. We develop a label-aware joint model to leverage label semantics for cross-lingual knowledge transfer.
  arXiv  Detail & Related papers  (2022-05-07T13:44:28Z)
• Cross-Lingual Dialogue Dataset Creation via Outline-Based Generation [70.81596088969378]
  Cross-lingual Outline-based Dialogue dataset (termed COD) enables natural language understanding. COD enables dialogue state tracking, and end-to-end dialogue modelling and evaluation in 4 diverse languages.
  arXiv  Detail & Related papers  (2022-01-31T18:11:21Z)
• From Masked Language Modeling to Translation: Non-English Auxiliary Tasks Improve Zero-shot Spoken Language Understanding [24.149299722716155]
  We introduce xSID, a new benchmark for cross-lingual Slot and Intent Detection in 13 languages from 6 language families, including a very low-resource dialect. We propose a joint learning approach, with English SLU training data and non-English auxiliary tasks from raw text, syntax and translation for transfer. Our results show that jointly learning the main tasks with masked language modeling is effective for slots, while machine translation transfer works best for intent classification.
  arXiv  Detail & Related papers  (2021-05-15T23:51:11Z)
• Crossing the Conversational Chasm: A Primer on Multilingual Task-Oriented Dialogue Systems [51.328224222640614]
  Current state-of-the-art ToD models based on large pretrained neural language models are data hungry. Data acquisition for ToD use cases is expensive and tedious.
  arXiv  Detail & Related papers  (2021-04-17T15:19:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.