Zero-shot Cross-lingual Conversational Semantic Role Labeling
- URL: http://arxiv.org/abs/2204.04914v1
- Date: Mon, 11 Apr 2022 07:29:39 GMT
- Title: Zero-shot Cross-lingual Conversational Semantic Role Labeling
- Authors: Han Wu, Haochen Tan, Kun Xu, Shuqi Liu, Lianwei Wu and Linqi Song
- Abstract summary: We present an effective approach to perform zero-shot cross-lingual CSRL.
Our model implicitly learns language-agnostic, conversational structure-aware and semantically rich representations with the hierarchical encoders.
We confirm the usefulness of CSRL to non-Chinese conversational tasks such as the question-in-context rewriting task in English and the multi-turn dialogue response generation tasks in English, German and Japanese.
- Score: 30.28068025468505
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: While conversational semantic role labeling (CSRL) has shown its usefulness
on Chinese conversational tasks, it is still under-explored in non-Chinese
languages due to the lack of multilingual CSRL annotations for the parser
training. To avoid expensive data collection and error-propagation of
translation-based methods, we present a simple but effective approach to
perform zero-shot cross-lingual CSRL. Our model implicitly learns
language-agnostic, conversational structure-aware and semantically rich
representations with the hierarchical encoders and elaborately designed
pre-training objectives. Experimental results show that our model outperforms
all baselines by large margins on two newly collected English CSRL test sets.
More importantly, we confirm the usefulness of CSRL to non-Chinese
conversational tasks such as the question-in-context rewriting task in English
and the multi-turn dialogue response generation tasks in English, German and
Japanese by incorporating the CSRL information into the downstream
conversation-based models. We believe this finding is significant and will
facilitate the research of non-Chinese dialogue tasks which suffer the problems
of ellipsis and anaphora.
Related papers
- Understanding Cross-Lingual Alignment -- A Survey [52.572071017877704]
Cross-lingual alignment is the meaningful similarity of representations across languages in multilingual language models.
We survey the literature of techniques to improve cross-lingual alignment, providing a taxonomy of methods and summarising insights from throughout the field.
arXiv Detail & Related papers (2024-04-09T11:39:53Z) - Decomposed Prompting: Unveiling Multilingual Linguistic Structure
Knowledge in English-Centric Large Language Models [12.700783525558721]
English-centric Large Language Models (LLMs) like GPT-3 and LLaMA display a remarkable ability to perform multilingual tasks.
This paper introduces the decomposed prompting approach to probe the linguistic structure understanding of these LLMs in sequence labeling tasks.
arXiv Detail & Related papers (2024-02-28T15:15:39Z) - Efficiently Aligned Cross-Lingual Transfer Learning for Conversational
Tasks using Prompt-Tuning [98.60739735409243]
Cross-lingual transfer of language models trained on high-resource languages like English has been widely studied for many NLP tasks.
We introduce XSGD for cross-lingual alignment pretraining, a parallel and large-scale multilingual conversation dataset.
To facilitate aligned cross-lingual representations, we develop an efficient prompt-tuning-based method for learning alignment prompts.
arXiv Detail & Related papers (2023-04-03T18:46:01Z) - Understanding Translationese in Cross-Lingual Summarization [106.69566000567598]
Cross-lingual summarization (MS) aims at generating a concise summary in a different target language.
To collect large-scale CLS data, existing datasets typically involve translation in their creation.
In this paper, we first confirm that different approaches of constructing CLS datasets will lead to different degrees of translationese.
arXiv Detail & Related papers (2022-12-14T13:41:49Z) - Conversational Semantic Role Labeling with Predicate-Oriented Latent
Graph [40.43625257213158]
We propose to automatically induce a predicate-oriented latent graph (POLar) with a predicate-centered Gaussian mechanism.
The POLar structure is then dynamically pruned and refined so as to best fit the task need.
We additionally introduce an effective dialogue-level pre-trained language model, CoDiaBERT, for better supporting multiple utterance sentences.
arXiv Detail & Related papers (2022-10-06T16:42:00Z) - Multi2WOZ: A Robust Multilingual Dataset and Conversational Pretraining
for Task-Oriented Dialog [67.20796950016735]
Multi2WOZ dataset spans four typologically diverse languages: Chinese, German, Arabic, and Russian.
We introduce a new framework for multilingual conversational specialization of pretrained language models (PrLMs) that aims to facilitate cross-lingual transfer for arbitrary downstream TOD tasks.
Our experiments show that, in most setups, the best performance entails the combination of (I) conversational specialization in the target language and (ii) few-shot transfer for the concrete TOD task.
arXiv Detail & Related papers (2022-05-20T18:35:38Z) - The Impact of Cross-Lingual Adjustment of Contextual Word
Representations on Zero-Shot Transfer [3.300216758849348]
Large multilingual language models such as mBERT or XLM-R enable zero-shot cross-lingual transfer in various IR and NLP tasks.
We propose a data- and compute-efficient method for cross-lingual adjustment of mBERT that uses a small parallel corpus to make embeddings of related words across languages similar to each other.
We experiment with a typologically diverse set of languages (Spanish, Russian, Vietnamese, and Hindi) and extend their original implementations to new tasks.
Our study reproduced gains in NLI for four languages, showed improved NER, XSR, and cross-lingual QA
arXiv Detail & Related papers (2022-04-13T15:28:43Z) - CSAGN: Conversational Structure Aware Graph Network for Conversational
Semantic Role Labeling [27.528361001332264]
We present a simple and effective architecture for CSRL which aims to address this problem.
Our model is based on a conversational structure-aware graph network which explicitly encodes the speaker dependent information.
arXiv Detail & Related papers (2021-09-23T07:47:28Z) - Crossing the Conversational Chasm: A Primer on Multilingual
Task-Oriented Dialogue Systems [51.328224222640614]
Current state-of-the-art ToD models based on large pretrained neural language models are data hungry.
Data acquisition for ToD use cases is expensive and tedious.
arXiv Detail & Related papers (2021-04-17T15:19:56Z) - Evaluating Multilingual Text Encoders for Unsupervised Cross-Lingual
Retrieval [51.60862829942932]
We present a systematic empirical study focused on the suitability of the state-of-the-art multilingual encoders for cross-lingual document and sentence retrieval tasks.
For sentence-level CLIR, we demonstrate that state-of-the-art performance can be achieved.
However, the peak performance is not met using the general-purpose multilingual text encoders off-the-shelf', but rather relying on their variants that have been further specialized for sentence understanding tasks.
arXiv Detail & Related papers (2021-01-21T00:15:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.