Analyzing the Effects of Reasoning Types on Cross-Lingual Transfer Performance

• URL: http://arxiv.org/abs/2110.02386v1
• Date: Tue, 5 Oct 2021 22:36:46 GMT
• Title: Analyzing the Effects of Reasoning Types on Cross-Lingual Transfer Performance
• Authors: Karthikeyan K, Aalok Sathe, Somak Aditya, Monojit Choudhury
• Abstract summary: Examples in Natural Language Inference (NLI) often pertain to various types of sub-tasks, requiring different kinds of reasoning. Certain types of reasoning have proven to be more difficult to learn in a monolingual context. We statistically observe interesting effects that the confluence of reasoning types and language similarities have on transfer performance.
• Score: 10.33152983955968
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multilingual language models achieve impressive zero-shot accuracies in many languages in complex tasks such as Natural Language Inference (NLI). Examples in NLI (and equivalent complex tasks) often pertain to various types of sub-tasks, requiring different kinds of reasoning. Certain types of reasoning have proven to be more difficult to learn in a monolingual context, and in the crosslingual context, similar observations may shed light on zero-shot transfer efficiency and few-shot sample selection. Hence, to investigate the effects of types of reasoning on transfer performance, we propose a category-annotated multilingual NLI dataset and discuss the challenges to scale monolingual annotations to multiple languages. We statistically observe interesting effects that the confluence of reasoning types and language similarities have on transfer performance.

Related papers

• Unknown Script: Impact of Script on Cross-Lingual Transfer [2.5398014196797605]
  Cross-lingual transfer has become an effective way of transferring knowledge between languages. We consider a case where the target language and its script are not part of the pre-trained model. Our findings reveal the importance of the tokenizer as a stronger factor than the shared script, language similarity, and model size.
  arXiv  Detail & Related papers  (2024-04-29T15:48:01Z)
• Event Extraction in Basque: Typologically motivated Cross-Lingual Transfer-Learning Analysis [18.25948580496853]
  Cross-lingual transfer-learning is widely used in Event Extraction for low-resource languages. This paper studies whether the typological similarity between source and target languages impacts the performance of cross-lingual transfer.
  arXiv  Detail & Related papers  (2024-04-09T15:35:41Z)
• Is Prompt-Based Finetuning Always Better than Vanilla Finetuning? Insights from Cross-Lingual Language Understanding [0.30586855806896046]
  We propose the ProFiT pipeline to investigate the cross-lingual capabilities of Prompt-based Finetuning. Our results reveal the effectiveness and versatility of prompt-based finetuning in cross-lingual language understanding.
  arXiv  Detail & Related papers  (2023-07-15T20:33:33Z)
• BUFFET: Benchmarking Large Language Models for Few-shot Cross-lingual Transfer [81.5984433881309]
  We introduce BUFFET, which unifies 15 diverse tasks across 54 languages in a sequence-to-sequence format. BUFFET is designed to establish a rigorous and equitable evaluation framework for few-shot cross-lingual transfer. Our findings reveal significant room for improvement in few-shot in-context cross-lingual transfer.
  arXiv  Detail & Related papers  (2023-05-24T08:06:33Z)
• Efficiently Aligned Cross-Lingual Transfer Learning for Conversational Tasks using Prompt-Tuning [98.60739735409243]
  Cross-lingual transfer of language models trained on high-resource languages like English has been widely studied for many NLP tasks. We introduce XSGD for cross-lingual alignment pretraining, a parallel and large-scale multilingual conversation dataset. To facilitate aligned cross-lingual representations, we develop an efficient prompt-tuning-based method for learning alignment prompts.
  arXiv  Detail & Related papers  (2023-04-03T18:46:01Z)
• Prompting Large Language Model for Machine Translation: A Case Study [87.88120385000666]
  We offer a systematic study on prompting strategies for machine translation. We examine factors for prompt template and demonstration example selection. We explore the use of monolingual data and the feasibility of cross-lingual, cross-domain, and sentence-to-document transfer learning.
  arXiv  Detail & Related papers  (2023-01-17T18:32:06Z)
• When is BERT Multilingual? Isolating Crucial Ingredients for Cross-lingual Transfer [15.578267998149743]
  We show that the absence of sub-word overlap significantly affects zero-shot transfer when languages differ in their word order. There is a strong correlation between transfer performance and word embedding alignment between languages. Our results call for focus in multilingual models on explicitly improving word embedding alignment between languages.
  arXiv  Detail & Related papers  (2021-10-27T21:25:39Z)
• Discovering Representation Sprachbund For Multilingual Pre-Training [139.05668687865688]
  We generate language representation from multilingual pre-trained models and conduct linguistic analysis. We cluster all the target languages into multiple groups and name each group as a representation sprachbund. Experiments are conducted on cross-lingual benchmarks and significant improvements are achieved compared to strong baselines.
  arXiv  Detail & Related papers  (2021-09-01T09:32:06Z)
• Gender Bias in Multilingual Embeddings and Cross-Lingual Transfer [101.58431011820755]
  We study gender bias in multilingual embeddings and how it affects transfer learning for NLP applications. We create a multilingual dataset for bias analysis and propose several ways for quantifying bias in multilingual representations.
  arXiv  Detail & Related papers  (2020-05-02T04:34:37Z)
• Bridging Linguistic Typology and Multilingual Machine Translation with Multi-View Language Representations [83.27475281544868]
  We use singular vector canonical correlation analysis to study what kind of information is induced from each source. We observe that our representations embed typology and strengthen correlations with language relationships. We then take advantage of our multi-view language vector space for multilingual machine translation, where we achieve competitive overall translation accuracy.
  arXiv  Detail & Related papers  (2020-04-30T16:25:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.