RuSentEval: Linguistic Source, Encoder Force!

• URL: http://arxiv.org/abs/2103.00573v2
• Date: Tue, 2 Mar 2021 11:40:25 GMT
• Title: RuSentEval: Linguistic Source, Encoder Force!
• Authors: Vladislav Mikhailov and Ekaterina Taktasheva and Elina Sigdel and Ekaterina Artemova
• Abstract summary: We introduce RuSentEval, an enhanced set of 14 probing tasks for Russian. We apply a combination of complementary probing methods to explore the distribution of various linguistic properties in five multilingual transformers. Our results provide intriguing findings that contradict the common understanding of how linguistic knowledge is represented.
• Score: 1.8160945635344525
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The success of pre-trained transformer language models has brought a great deal of interest on how these models work, and what they learn about language. However, prior research in the field is mainly devoted to English, and little is known regarding other languages. To this end, we introduce RuSentEval, an enhanced set of 14 probing tasks for Russian, including ones that have not been explored yet. We apply a combination of complementary probing methods to explore the distribution of various linguistic properties in five multilingual transformers for two typologically contrasting languages -- Russian and English. Our results provide intriguing findings that contradict the common understanding of how linguistic knowledge is represented, and demonstrate that some properties are learned in a similar manner despite the language differences.

Related papers

• IndicSentEval: How Effectively do Multilingual Transformer Models encode Linguistic Properties for Indic Languages? [14.77467551053299]
  Transformer-based models have revolutionized the field of natural language processing. How robust are these models in encoding linguistic properties when faced with perturbations in the input text? In this paper, we investigate similar questions regarding encoding capability and robustness for 8 linguistic properties across 13 different perturbations in 6 Indic languages.
  arXiv  Detail & Related papers  (2024-10-03T15:50:08Z)
• Decoupled Vocabulary Learning Enables Zero-Shot Translation from Unseen Languages [55.157295899188476]
  neural machine translation systems learn to map sentences of different languages into a common representation space. In this work, we test this hypothesis by zero-shot translating from unseen languages. We demonstrate that this setup enables zero-shot translation from entirely unseen languages.
  arXiv  Detail & Related papers  (2024-08-05T07:58:58Z)
• A Computational Model for the Assessment of Mutual Intelligibility Among Closely Related Languages [1.5773159234875098]
  Closely related languages show linguistic similarities that allow speakers of one language to understand speakers of another language without having actively learned it. Mutual intelligibility varies in degree and is typically tested in psycholinguistic experiments. We propose a computer-assisted method using the Linear Discriminative Learner to approximate the cognitive processes by which humans learn languages.
  arXiv  Detail & Related papers  (2024-02-05T11:32:13Z)
• Establishing Interlingua in Multilingual Language Models [0.0]
  We show that different languages do converge to a shared space in large multilingual language models. We extend our analysis to 28 diverse languages and find that the interlingual space exhibits a particular structure similar to the linguistic relatedness of languages.
  arXiv  Detail & Related papers  (2021-09-02T20:53:14Z)
• Discovering Representation Sprachbund For Multilingual Pre-Training [139.05668687865688]
  We generate language representation from multilingual pre-trained models and conduct linguistic analysis. We cluster all the target languages into multiple groups and name each group as a representation sprachbund. Experiments are conducted on cross-lingual benchmarks and significant improvements are achieved compared to strong baselines.
  arXiv  Detail & Related papers  (2021-09-01T09:32:06Z)
• Morph Call: Probing Morphosyntactic Content of Multilingual Transformers [2.041108289731398]
  We present Morph Call, a suite of 46 probing tasks for four Indo-European languages of different morphology: English, French, German and Russian. We use a combination of neuron-, layer- and representation-level introspection techniques to analyze the morphosyntactic content of four multilingual transformers. The results show that fine-tuning for POS-tagging can improve and decrease the probing performance and change how morphosyntactic knowledge is distributed across the model.
  arXiv  Detail & Related papers  (2021-04-26T19:53:00Z)
• AM2iCo: Evaluating Word Meaning in Context across Low-ResourceLanguages with Adversarial Examples [51.048234591165155]
  We present AM2iCo, Adversarial and Multilingual Meaning in Context. It aims to faithfully assess the ability of state-of-the-art (SotA) representation models to understand the identity of word meaning in cross-lingual contexts. Results reveal that current SotA pretrained encoders substantially lag behind human performance.
  arXiv  Detail & Related papers  (2021-04-17T20:23:45Z)
• Are Multilingual Models Effective in Code-Switching? [57.78477547424949]
  We study the effectiveness of multilingual language models to understand their capability and adaptability to the mixed-language setting. Our findings suggest that pre-trained multilingual models do not necessarily guarantee high-quality representations on code-switching.
  arXiv  Detail & Related papers  (2021-03-24T16:20:02Z)
• Verb Knowledge Injection for Multilingual Event Processing [50.27826310460763]
  We investigate whether injecting explicit information on verbs' semantic-syntactic behaviour improves the performance of LM-pretrained Transformers. We first demonstrate that injecting verb knowledge leads to performance gains in English event extraction. We then explore the utility of verb adapters for event extraction in other languages.
  arXiv  Detail & Related papers  (2020-12-31T03:24:34Z)
• Bridging Linguistic Typology and Multilingual Machine Translation with Multi-View Language Representations [83.27475281544868]
  We use singular vector canonical correlation analysis to study what kind of information is induced from each source. We observe that our representations embed typology and strengthen correlations with language relationships. We then take advantage of our multi-view language vector space for multilingual machine translation, where we achieve competitive overall translation accuracy.
  arXiv  Detail & Related papers  (2020-04-30T16:25:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.