That Slepen Al the Nyght with Open Ye! Cross-era Sequence Segmentation with Switch-memory

• URL: http://arxiv.org/abs/2209.02967v1
• Date: Wed, 7 Sep 2022 07:21:23 GMT
• Title: That Slepen Al the Nyght with Open Ye! Cross-era Sequence Segmentation with Switch-memory
• Abstract summary: Cross-era learning framework for Chinese word segmentation (CWS) CrossWISE uses the Switch-memory (SM) module to incorporate era-specific linguistic knowledge. Experiments on four corpora from different eras show that the performance of each corpus significantly improves.
• Score: 8.569804490994219
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The evolution of language follows the rule of gradual change. Grammar, vocabulary, and lexical semantic shifts take place over time, resulting in a diachronic linguistic gap. As such, a considerable amount of texts are written in languages of different eras, which creates obstacles for natural language processing tasks, such as word segmentation and machine translation. Although the Chinese language has a long history, previous Chinese natural language processing research has primarily focused on tasks within a specific era. Therefore, we propose a cross-era learning framework for Chinese word segmentation (CWS), CROSSWISE, which uses the Switch-memory (SM) module to incorporate era-specific linguistic knowledge. Experiments on four corpora from different eras show that the performance of each corpus significantly improves. Further analyses also demonstrate that the SM can effectively integrate the knowledge of the eras into the neural network.

Related papers

• ChronoLens: Measuring Language Change Across Time, Languages, and Linguistic Levels [30.58256014365214]
  We show how the magnitude and direction of change vary across linguistic levels, languages, and historical periods within a single analytical space.<n>We introduce ChronoLens, a framework that combines frozen multilingual language models, feature-aligned crosscoders, and post-hoc linguistic interventions.
  arXiv  Detail & Related papers  (2026-08-04T11:49:16Z)
• Tokenizing Crosslingual Homographs [8.254230288283258]
  Multilingual language models rely on shared subword vocabularies to represent multiple languages within a limited number of token units.<n>We investigate this limitation through cross-lingual homographs and false friends.<n>We propose a simple tokenizer-level intervention based on language cues: language-specific characters replacing initial characters of shared-vocabulary words.
  arXiv  Detail & Related papers  (2026-07-20T08:37:31Z)
• Integrating Linguistics and AI: Morphological Analysis and Corpus development of Endangered Toto Language of West Bengal [0.6089496237595778]
  This paper is a part of a project which aims to develop a trilingual (Toto-Bangla-English) language learning application.<n>It aims to digitally archive and promote the endangered Toto language of West Bengal, India.
  arXiv  Detail & Related papers  (2025-10-26T11:22:46Z)
• UniPSDA: Unsupervised Pseudo Semantic Data Augmentation for Zero-Shot Cross-Lingual Natural Language Understanding [31.272603877215733]
  Cross-lingual representation learning transfers knowledge from resource-rich data to resource-scarce ones to improve the semantic understanding abilities of different languages. We propose an Unsupervised Pseudo Semantic Data Augmentation (UniPSDA) mechanism for cross-lingual natural language understanding to enrich the training data without human interventions.
  arXiv  Detail & Related papers  (2024-06-24T07:27:01Z)
• Wav2Gloss: Generating Interlinear Glossed Text from Speech [78.64412090339044]
  We propose Wav2Gloss, a task in which four linguistic annotation components are extracted automatically from speech. We provide various baselines to lay the groundwork for future research on Interlinear Glossed Text generation from speech.
  arXiv  Detail & Related papers  (2024-03-19T21:45:29Z)
• Fine-Tuned Self-Supervised Speech Representations for Language Diarization in Multilingual Code-Switched Speech [4.39549503760707]
  We develop a continuous multilingual language diarizer using fine-tuned speech representations extracted from a large self-supervised architecture (WavLM) We experiment with a code-switched corpus consisting of five South African languages (isiZulu, isiXa, Setswana, Sesotho and English)
  arXiv  Detail & Related papers  (2023-12-15T09:40:41Z)
• Cross-Lingual Knowledge Editing in Large Language Models [73.12622532088564]
  Knowledge editing has been shown to adapt large language models to new knowledge without retraining from scratch. It is still unknown the effect of source language editing on a different target language. We first collect a large-scale cross-lingual synthetic dataset by translating ZsRE from English to Chinese.
  arXiv  Detail & Related papers  (2023-09-16T11:07:52Z)
• Robust Open-Set Spoken Language Identification and the CU MultiLang Dataset [2.048226951354646]
  Open-set spoken language identification systems can detect when an input exhibits none of the original languages. We implement a novel approach to open-set spoken language identification that uses MFCC and pitch features. We present a spoken language identification system that achieves 91.76% accuracy on trained languages and has the capability to adapt to unknown languages on the fly.
  arXiv  Detail & Related papers  (2023-08-29T00:44:27Z)
• An Empirical Revisiting of Linguistic Knowledge Fusion in Language Understanding Tasks [33.765874588342285]
  Infusing language models with syntactic or semantic knowledge from structural linguistic priors has shown improvements on many language understanding tasks. We conduct empirical study of replacing parsed graphs or trees with trivial ones for tasks in the GLUE benchmark. It reveals that the gains might not be significantly attributed to explicit linguistic priors but rather to more feature interactions brought by fusion layers.
  arXiv  Detail & Related papers  (2022-10-24T07:47:32Z)
• LAE: Language-Aware Encoder for Monolingual and Multilingual ASR [87.74794847245536]
  A novel language-aware encoder (LAE) architecture is proposed to handle both situations by disentangling language-specific information. Experiments conducted on Mandarin-English code-switched speech suggest that the proposed LAE is capable of discriminating different languages in frame-level.
  arXiv  Detail & Related papers  (2022-06-05T04:03:12Z)
• Cross-Lingual Ability of Multilingual Masked Language Models: A Study of Language Structure [54.01613740115601]
  We study three language properties: constituent order, composition and word co-occurrence. Our main conclusion is that the contribution of constituent order and word co-occurrence is limited, while the composition is more crucial to the success of cross-linguistic transfer.
  arXiv  Detail & Related papers  (2022-03-16T07:09:35Z)
• When is BERT Multilingual? Isolating Crucial Ingredients for Cross-lingual Transfer [15.578267998149743]
  We show that the absence of sub-word overlap significantly affects zero-shot transfer when languages differ in their word order. There is a strong correlation between transfer performance and word embedding alignment between languages. Our results call for focus in multilingual models on explicitly improving word embedding alignment between languages.
  arXiv  Detail & Related papers  (2021-10-27T21:25:39Z)
• VECO: Variable and Flexible Cross-lingual Pre-training for Language Understanding and Generation [77.82373082024934]
  We plug a cross-attention module into the Transformer encoder to explicitly build the interdependence between languages. It can effectively avoid the degeneration of predicting masked words only conditioned on the context in its own language. The proposed cross-lingual model delivers new state-of-the-art results on various cross-lingual understanding tasks of the XTREME benchmark.
  arXiv  Detail & Related papers  (2020-10-30T03:41:38Z)
• Bridging Linguistic Typology and Multilingual Machine Translation with Multi-View Language Representations [83.27475281544868]
  We use singular vector canonical correlation analysis to study what kind of information is induced from each source. We observe that our representations embed typology and strengthen correlations with language relationships. We then take advantage of our multi-view language vector space for multilingual machine translation, where we achieve competitive overall translation accuracy.
  arXiv  Detail & Related papers  (2020-04-30T16:25:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.