Cross-lingual Semantic Role Labeling with Model Transfer

• URL: http://arxiv.org/abs/2008.10284v1
• Date: Mon, 24 Aug 2020 09:37:45 GMT
• Title: Cross-lingual Semantic Role Labeling with Model Transfer
• Authors: Hao Fei and Meishan Zhang and Fei Li and Donghong Ji
• Abstract summary: Cross-lingual semantic role labeling can be achieved by model transfer under the help of universal features. We propose an end-to-end SRL model that incorporates a variety of universal features and transfer methods.
• Score: 49.85316125365497
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Prior studies show that cross-lingual semantic role labeling (SRL) can be achieved by model transfer under the help of universal features. In this paper, we fill the gap of cross-lingual SRL by proposing an end-to-end SRL model that incorporates a variety of universal features and transfer methods. We study both the bilingual transfer and multi-source transfer, under gold or machine-generated syntactic inputs, pre-trained high-order abstract features, and contextualized multilingual word representations. Experimental results on the Universal Proposition Bank corpus indicate that performances of the cross-lingual SRL can vary by leveraging different cross-lingual features. In addition, whether the features are gold-standard also has an impact on performances. Precisely, we find that gold syntax features are much more crucial for cross-lingual SRL, compared with the automatically-generated ones. Moreover, universal dependency structure features are able to give the best help, and both pre-trained high-order features and contextualized word representations can further bring significant improvements.

Related papers

• ShifCon: Enhancing Non-Dominant Language Capabilities with a Shift-based Contrastive Framework [79.72910257530795]
  ShifCon is a Shift-based Contrastive framework that aligns the internal forward process of other languages toward that of the dominant one. It shifts the representations of non-dominant languages into the dominant language subspace, allowing them to access relatively rich information encoded in the model parameters. Experiments demonstrate that our ShifCon framework significantly enhances the performance of non-dominant languages.
  arXiv  Detail & Related papers  (2024-10-25T10:28:59Z)
• Cross-lingual Back-Parsing: Utterance Synthesis from Meaning Representation for Zero-Resource Semantic Parsing [6.074150063191985]
  Cross-Lingual Back-Parsing is a novel data augmentation methodology designed to enhance cross-lingual transfer for semantic parsing. Our methodology effectively performs cross-lingual data augmentation in challenging zero-resource settings.
  arXiv  Detail & Related papers  (2024-10-01T08:53:38Z)
• Self-Augmentation Improves Zero-Shot Cross-Lingual Transfer [92.80671770992572]
  Cross-lingual transfer is a central task in multilingual NLP. Earlier efforts on this task use parallel corpora, bilingual dictionaries, or other annotated alignment data. We propose a simple yet effective method, SALT, to improve the zero-shot cross-lingual transfer.
  arXiv  Detail & Related papers  (2023-09-19T19:30:56Z)
• VECO 2.0: Cross-lingual Language Model Pre-training with Multi-granularity Contrastive Learning [56.47303426167584]
  We propose a cross-lingual pre-trained model VECO2.0 based on contrastive learning with multi-granularity alignments. Specifically, the sequence-to-sequence alignment is induced to maximize the similarity of the parallel pairs and minimize the non-parallel pairs. token-to-token alignment is integrated to bridge the gap between synonymous tokens excavated via the thesaurus dictionary from the other unpaired tokens in a bilingual instance.
  arXiv  Detail & Related papers  (2023-04-17T12:23:41Z)
• DiTTO: A Feature Representation Imitation Approach for Improving Cross-Lingual Transfer [15.062937537799005]
  languages as domains for improving zero-shot transfer. We show that our approach, DiTTO, significantly outperforms the standard zero-shot fine-tuning method. Our model enables better cross-lingual transfer than standard fine-tuning methods, even in the few-shot setting.
  arXiv  Detail & Related papers  (2023-03-04T08:42:50Z)
• Transition-based Semantic Role Labeling with Pointer Networks [0.40611352512781856]
  We propose the first transition-based SRL approach that is capable of completely processing an input sentence in a single left-to-right pass. Thanks to our implementation based on Pointer Networks, full SRL can be accurately and efficiently done in $O(n2)$, achieving the best performance to date on the majority of languages from the CoNLL-2009 shared task.
  arXiv  Detail & Related papers  (2022-05-20T08:38:44Z)
• VECO: Variable and Flexible Cross-lingual Pre-training for Language Understanding and Generation [77.82373082024934]
  We plug a cross-attention module into the Transformer encoder to explicitly build the interdependence between languages. It can effectively avoid the degeneration of predicting masked words only conditioned on the context in its own language. The proposed cross-lingual model delivers new state-of-the-art results on various cross-lingual understanding tasks of the XTREME benchmark.
  arXiv  Detail & Related papers  (2020-10-30T03:41:38Z)
• X-SRL: A Parallel Cross-Lingual Semantic Role Labeling Dataset [18.389328059694037]
  In this work, we propose a method to automatically construct an SRL corpus that is parallel in four languages: English, French, German, Spanish, with unified predicate and role annotations that are fully comparable across languages. We include human-validated test sets that we use to measure the projection quality, and show that projection is denser and more precise than a strong baseline. Finally, we train different SOTA models on our novel corpus for mono- and multilingual SRL, showing that the multilingual annotations improve performance especially for the weaker languages.
  arXiv  Detail & Related papers  (2020-10-05T13:34:20Z)
• A Study of Cross-Lingual Ability and Language-specific Information in Multilingual BERT [60.9051207862378]
  multilingual BERT works remarkably well on cross-lingual transfer tasks. Datasize and context window size are crucial factors to the transferability. There is a computationally cheap but effective approach to improve the cross-lingual ability of multilingual BERT.
  arXiv  Detail & Related papers  (2020-04-20T11:13:16Z)
• Cross-Lingual Semantic Role Labeling with High-Quality Translated Training Corpus [41.031187560839555]
  Cross-lingual semantic role labeling is one promising way to address the problem. We propose a novel alternative based on corpus translation, constructing high-quality training datasets for the target languages. Experimental results on Universal Proposition Bank show that the translation-based method is highly effective.
  arXiv  Detail & Related papers  (2020-04-14T04:16:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.