Related papers: Learning thresholds lead to stable language coexistence

Learning thresholds lead to stable language coexistence

URL: http://arxiv.org/abs/2406.14522v1
Date: Fri, 14 Jun 2024 14:24:02 GMT
Title: Learning thresholds lead to stable language coexistence
Authors: Mikhail V. Tamm, Els Heinsalu, Stefano Scialla, Marco Patriarca,
Abstract summary: We introduce a language competition model that incorporates the effects of memory and learning on the language shift dynamics. On a coarse grained time scale, the effects of memory and learning can be expressed as thresholds on the speakers fractions.
Score: 0.0
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
Abstract: We introduce a language competition model that incorporates the effects of memory and learning on the language shift dynamics, using the Abrams-Strogatz model as a starting point. On a coarse grained time scale, the effects of memory and learning can be expressed as thresholds on the speakers fractions. In its simplest form, the resulting model is exactly solvable. Besides the consensus on one of the two languages, the model describes additional equilibrium states that are not present in the Abrams-Strogatz model: a stable coexistence of the two languages, if both thresholds are low enough, so that the language shift processes in the two opposite directions compensate each other, and a frozen state coinciding with the initial state, when both thresholds are too high for any language shift to take place. We show numerically that these results are preserved for threshold functions of a more general shape.

Related papers

Modeling language contact with the Iterated Learning Model [0.0]
Iterated learning models are agent-based models of language change. A recently introduced type of iterated learning model, the Semi-Supervised ILM is used to simulate language contact.
arXiv Detail & Related papers (2024-06-11T01:43:23Z)
Transparency Helps Reveal When Language Models Learn Meaning [71.96920839263457]
Our systematic experiments with synthetic data reveal that, with languages where all expressions have context-independent denotations, both autoregressive and masked language models learn to emulate semantic relations between expressions. Turning to natural language, our experiments with a specific phenomenon -- referential opacity -- add to the growing body of evidence that current language models do not well-represent natural language semantics.
arXiv Detail & Related papers (2022-10-14T02:35:19Z)
Shapley Head Pruning: Identifying and Removing Interference in Multilingual Transformers [54.4919139401528]
We show that it is possible to reduce interference by identifying and pruning language-specific parameters. We show that removing identified attention heads from a fixed model improves performance for a target language on both sentence classification and structural prediction.
arXiv Detail & Related papers (2022-10-11T18:11:37Z)
Quark: Controllable Text Generation with Reinforced Unlearning [68.07749519374089]
Large-scale language models often learn behaviors that are misaligned with user expectations. We introduce Quantized Reward Konditioning (Quark), an algorithm for optimizing a reward function that quantifies an (un)wanted property. For unlearning toxicity, negative sentiment, and repetition, our experiments show that Quark outperforms both strong baselines and state-of-the-art reinforcement learning methods.
arXiv Detail & Related papers (2022-05-26T21:11:51Z)
Improving Neural Cross-Lingual Summarization via Employing Optimal Transport Distance for Knowledge Distillation [8.718749742587857]
Cross-lingual summarization models rely on the self-attention mechanism to attend among tokens in two languages. We propose a novel Knowledge-Distillation-based framework for Cross-Lingual Summarization. Our method outperforms state-of-the-art models under both high and low-resourced settings.
arXiv Detail & Related papers (2021-12-07T03:45:02Z)
Finding Fuzziness in Neural Network Models of Language Processing [0.2538209532048867]
We test the extent to which models trained to capture the distributional statistics of language show correspondence to fuzzy-membership patterns. We find the model to show patterns that are similar to classical fuzzy-set theoretic formulations of linguistic hedges, albeit with a substantial amount of noise.
arXiv Detail & Related papers (2021-04-22T01:06:14Z)
Comparison of Interactive Knowledge Base Spelling Correction Models for Low-Resource Languages [81.90356787324481]
Spelling normalization for low resource languages is a challenging task because the patterns are hard to predict. This work shows a comparison of a neural model and character language models with varying amounts on target language data. Our usage scenario is interactive correction with nearly zero amounts of training examples, improving models as more data is collected.
arXiv Detail & Related papers (2020-10-20T17:31:07Z)
Cross-lingual Spoken Language Understanding with Regularized Representation Alignment [71.53159402053392]
We propose a regularization approach to align word-level and sentence-level representations across languages without any external resource. Experiments on the cross-lingual spoken language understanding task show that our model outperforms current state-of-the-art methods in both few-shot and zero-shot scenarios.
arXiv Detail & Related papers (2020-09-30T08:56:53Z)
Multi-timescale Representation Learning in LSTM Language Models [69.98840820213937]
Language models must capture statistical dependencies between words at timescales ranging from very short to very long. We derived a theory for how the memory gating mechanism in long short-term memory language models can capture power law decay. Experiments showed that LSTM language models trained on natural English text learn to approximate this theoretical distribution.
arXiv Detail & Related papers (2020-09-27T02:13:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.