EuskañolDS: A Naturally Sourced Corpus for Basque-Spanish Code-Switching

• URL: http://arxiv.org/abs/2502.03188v1
• Date: Wed, 05 Feb 2025 14:04:42 GMT
• Title: EuskañolDS: A Naturally Sourced Corpus for Basque-Spanish Code-Switching
• Abstract summary: We introduce a first approach to develop a naturally sourced corpus for Basque-Spanish code-switching.<n>Our methodology consists of identifying CS texts from previously available corpora using language identification models.<n>We present the properties of our corpus and make it available under the name EuskanolDS.
• Score: 15.839595122482196
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Code-switching (CS) remains a significant challenge in Natural Language Processing (NLP), mainly due a lack of relevant data. In the context of the contact between the Basque and Spanish languages in the north of the Iberian Peninsula, CS frequently occurs in both formal and informal spontaneous interactions. However, resources to analyse this phenomenon and support the development and evaluation of models capable of understanding and generating code-switched language for this language pair are almost non-existent. We introduce a first approach to develop a naturally sourced corpus for Basque-Spanish code-switching. Our methodology consists of identifying CS texts from previously available corpora using language identification models, which are then manually validated to obtain a reliable subset of CS instances. We present the properties of our corpus and make it available under the name Euska\~nolDS.

Related papers

• Exploring NLP Benchmarks in an Extremely Low-Resource Setting [21.656551146954587]
  This paper focuses on Ladin, an endangered Romance language, specifically targeting the Val Badia variant.<n>We create synthetic datasets for sentiment analysis and multiple-choice question answering (MCQA) by translating monolingual Italian data.
  arXiv  Detail & Related papers  (2025-09-04T07:41:23Z)
• Lost in Variation? Evaluating NLI Performance in Basque and Spanish Geographical Variants [7.160574787275442]
  We evaluate the capacity of current language technologies to understand Basque and Spanish language varieties.<n>We use Natural Language Inference (NLI) as a pivot task and introduce a novel, manually-curated parallel dataset.
  arXiv  Detail & Related papers  (2025-06-18T08:20:19Z)
• Investigating and Scaling up Code-Switching for Multilingual Language Model Pre-Training [58.696660064190475]
  We find that the existence of code-switching, alternating between different languages within a context, is key to multilingual capabilities. To better explore the power of code-switching for language alignment during pre-training, we investigate the strategy of synthetic code-switching.
  arXiv  Detail & Related papers  (2025-04-02T15:09:58Z)
• Conditioning LLMs to Generate Code-Switched Text: A Methodology Grounded in Naturally Occurring Data [21.240439045909724]
  Code-switching (CS) is still a critical challenge in Natural Language Processing (NLP) This paper presents a novel methodology to generate CS data using Large Language Models (LLMs) We propose back-translating natural CS sentences into monolingual English, and using the resulting parallel corpus to fine-tune LLMs to turn monolingual sentences into CS.
  arXiv  Detail & Related papers  (2025-02-18T15:04:13Z)
• Predictability and Causality in Spanish and English Natural Language Generation [6.817247544942709]
  This paper compares causal and non-causal language modeling for English and Spanish. According to this experiment, Spanish is more predictable than English given a non-causal context. These insights support further research in NLG in Spanish using bidirectional transformer language models.
  arXiv  Detail & Related papers  (2024-08-26T14:09:28Z)
• Code-Mixed Probes Show How Pre-Trained Models Generalise On Code-Switched Text [1.9185059111021852]
  We investigate how pre-trained Language Models handle code-switched text in three dimensions. Our findings reveal that pre-trained language models are effective in generalising to code-switched text.
  arXiv  Detail & Related papers  (2024-03-07T19:46:03Z)
• Spanish Resource Grammar version 2023 [12.009437358109407]
  We present the latest version of the Spanish Resource Grammar (SRG) Such grammars encode a complex set of hypotheses about syntax making them a resource for empirical testing of linguistic theory. This version of the SRG uses the recent version of the Freeling morphological and is released along with an automatically created, manually verified treebank of 2,291 sentences.
  arXiv  Detail & Related papers  (2023-09-23T09:24:05Z)
• Towards Zero-shot Language Modeling [90.80124496312274]
  We construct a neural model that is inductively biased towards learning human languages. We infer this distribution from a sample of typologically diverse training languages. We harness additional language-specific side information as distant supervision for held-out languages.
  arXiv  Detail & Related papers  (2021-08-06T23:49:18Z)
• Constructing a Family Tree of Ten Indo-European Languages with Delexicalized Cross-linguistic Transfer Patterns [57.86480614673034]
  We formalize the delexicalized transfer as interpretable tree-to-string and tree-to-tree patterns. This allows us to quantitatively probe cross-linguistic transfer and extend inquiries of Second Language Acquisition.
  arXiv  Detail & Related papers  (2020-07-17T15:56:54Z)
• Style Variation as a Vantage Point for Code-Switching [54.34370423151014]
  Code-Switching (CS) is a common phenomenon observed in several bilingual and multilingual communities. We present a novel vantage point of CS to be style variations between both the participating languages. We propose a two-stage generative adversarial training approach where the first stage generates competitive negative examples for CS and the second stage generates more realistic CS sentences.
  arXiv  Detail & Related papers  (2020-05-01T15:53:16Z)
• XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning [68.57658225995966]
  Cross-lingual Choice of Plausible Alternatives (XCOPA) is a typologically diverse multilingual dataset for causal commonsense reasoning in 11 languages. We evaluate a range of state-of-the-art models on this novel dataset, revealing that the performance of current methods falls short compared to translation-based transfer.
  arXiv  Detail & Related papers  (2020-05-01T12:22:33Z)
• Rnn-transducer with language bias for end-to-end Mandarin-English code-switching speech recognition [58.105818353866354]
  We propose an improved recurrent neural network transducer (RNN-T) model with language bias to alleviate the problem. We use the language identities to bias the model to predict the CS points. This promotes the model to learn the language identity information directly from transcription, and no additional LID model is needed.
  arXiv  Detail & Related papers  (2020-02-19T12:01:33Z)
• CoVoST: A Diverse Multilingual Speech-To-Text Translation Corpus [57.641761472372814]
  CoVoST is a multilingual speech-to-text translation corpus from 11 languages into English. It diversified with over 11,000 speakers and over 60 accents. CoVoST is released under CC0 license and free to use.
  arXiv  Detail & Related papers  (2020-02-04T14:35:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.