Towards Instance-Level Parser Selection for Cross-Lingual Transfer of Dependency Parsers

• URL: http://arxiv.org/abs/2004.07642v1
• Date: Thu, 16 Apr 2020 13:18:55 GMT
• Title: Towards Instance-Level Parser Selection for Cross-Lingual Transfer of Dependency Parsers
• Authors: Robert Litschko, Ivan Vuli\'c, \v{Z}eljko Agi\'c, Goran Glava\v{s}
• Abstract summary: We argue for a novel cross-lingual transfer paradigm: instance-level selection (ILPS) We present a proof-of-concept study focused on instance-level selection in the framework of delexicalized transfer.
• Score: 59.345145623931636
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Current methods of cross-lingual parser transfer focus on predicting the best parser for a low-resource target language globally, that is, "at treebank level". In this work, we propose and argue for a novel cross-lingual transfer paradigm: instance-level parser selection (ILPS), and present a proof-of-concept study focused on instance-level selection in the framework of delexicalized parser transfer. We start from an empirical observation that different source parsers are the best choice for different Universal POS sequences in the target language. We then propose to predict the best parser at the instance level. To this end, we train a supervised regression model, based on the Transformer architecture, to predict parser accuracies for individual POS-sequences. We compare ILPS against two strong single-best parser selection baselines (SBPS): (1) a model that compares POS n-gram distributions between the source and target languages (KL) and (2) a model that selects the source based on the similarity between manually created language vectors encoding syntactic properties of languages (L2V). The results from our extensive evaluation, coupling 42 source parsers and 20 diverse low-resource test languages, show that ILPS outperforms KL and L2V on 13/20 and 14/20 test languages, respectively. Further, we show that by predicting the best parser "at the treebank level" (SBPS), using the aggregation of predictions from our instance-level model, we outperform the same baselines on 17/20 and 16/20 test languages.

Related papers

• Multilingual Contrastive Decoding via Language-Agnostic Layers Skipping [60.458273797431836]
  Decoding by contrasting layers (DoLa) is designed to improve the generation quality of large language models. We find that this approach does not work well on non-English tasks. Inspired by previous interpretability work on language transition during the model's forward pass, we propose an improved contrastive decoding algorithm.
  arXiv  Detail & Related papers  (2024-07-15T15:14:01Z)
• To token or not to token: A Comparative Study of Text Representations for Cross-Lingual Transfer [23.777874316083984]
  We propose a scoring Language Quotient metric capable of providing a weighted representation of both zero-shot and few-shot evaluation combined. Our analysis reveals that image-based models excel in cross-lingual transfer when languages are closely related and share visually similar scripts. In dependency parsing tasks where word relationships play a crucial role, models with their character-level focus, outperform others.
  arXiv  Detail & Related papers  (2023-10-12T06:59:10Z)
• A Simple and Strong Baseline for End-to-End Neural RST-style Discourse Parsing [44.72809363746258]
  This paper explores a strong baseline by integrating existing simple parsing strategies, top-down and bottom-up, with various transformer-based pre-trained language models. The experimental results obtained from two benchmark datasets demonstrate that the parsing performance relies on the pretrained language models rather than the parsing strategies.
  arXiv  Detail & Related papers  (2022-10-15T18:38:08Z)
• BenchCLAMP: A Benchmark for Evaluating Language Models on Syntactic and Semantic Parsing [55.058258437125524]
  We introduce BenchCLAMP, a Benchmark to evaluate Constrained LAnguage Model Parsing. We benchmark eight language models, including two GPT-3 variants available only through an API. Our experiments show that encoder-decoder pretrained language models can achieve similar performance or surpass state-of-the-art methods for syntactic and semantic parsing when the model output is constrained to be valid.
  arXiv  Detail & Related papers  (2022-06-21T18:34:11Z)
• Unsupervised and Few-shot Parsing from Pretrained Language Models [56.33247845224995]
  We propose an Unsupervised constituent Parsing model that calculates an Out Association score solely based on the self-attention weight matrix learned in a pretrained language model. We extend the unsupervised models to few-shot parsing models that use a few annotated trees to learn better linear projection matrices for parsing. Our few-shot parsing model FPIO trained with only 20 annotated trees outperforms a previous few-shot parsing method trained with 50 annotated trees.
  arXiv  Detail & Related papers  (2022-06-10T10:29:15Z)
• EAG: Extract and Generate Multi-way Aligned Corpus for Complete Multi-lingual Neural Machine Translation [63.88541605363555]
  "Extract and Generate" (EAG) is a two-step approach to construct large-scale and high-quality multi-way aligned corpus from bilingual data. We first extract candidate aligned examples by pairing the bilingual examples from different language pairs with highly similar source or target sentences. We then generate the final aligned examples from the candidates with a well-trained generation model.
  arXiv  Detail & Related papers  (2022-03-04T08:21:27Z)
• Substructure Distribution Projection for Zero-Shot Cross-Lingual Dependency Parsing [55.69800855705232]
  SubDP is a technique that projects a distribution over structures in one domain to another, by projecting substructure distributions separately. We evaluate SubDP on zero-shot cross-lingual dependency parsing, taking dependency arcs as substructures.
  arXiv  Detail & Related papers  (2021-10-16T10:12:28Z)
• TGIF: Tree-Graph Integrated-Format Parser for Enhanced UD with Two-Stage Generic- to Individual-Language Finetuning [18.71574180551552]
  We present our contribution to the IWPT 2021 shared task on parsing into enhanced Universal Dependencies. Our main system component is a hybrid tree-graph that integrates predictions of spanning trees for the enhanced graphs with additional graph edges not present in the spanning trees.
  arXiv  Detail & Related papers  (2021-07-14T18:00:08Z)
• Applying Occam's Razor to Transformer-Based Dependency Parsing: What Works, What Doesn't, and What is Really Necessary [9.347252855045125]
  We study the choice of pre-trained embeddings and whether they use LSTM layers in graph-based dependency schemes. We propose a simple but widely applicable architecture and configuration, achieving new state-of-the-art results (in terms of LAS) for 10 out of 12 diverse languages.
  arXiv  Detail & Related papers  (2020-10-23T22:58:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.