On the Evolution of Word Order

• URL: http://arxiv.org/abs/2101.09579v1
• Date: Sat, 23 Jan 2021 20:30:17 GMT
• Title: On the Evolution of Word Order
• Authors: Idan Rejwan and Avi Caciularu
• Abstract summary: We show that an optimal language is one with fixed word order. We also show that adding information to the sentence, such as case markers and noun-verb distinction, reduces the need for fixed word order.
• Score: 7.2610922684683645
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Most natural languages have a predominant or fixed word order. For example, in English, the word order used most often is Subject-Verb-Object. This work attempts to explain this phenomena as well as other typological findings regarding word order from a functional perspective. That is, we target the question of whether fixed word order gives a functional advantage, that may explain why these languages are common. To this end, we consider an evolutionary model of language and show, both theoretically and using a genetic algorithm-based simulation, that an optimal language is one with fixed word order. We also show that adding information to the sentence, such as case markers and noun-verb distinction, reduces the need for fixed word order, in accordance with the typological findings.

Related papers

• Word Order's Impacts: Insights from Reordering and Generation Analysis [9.0720895802828]
  Existing works have studied the impacts of the order of words within natural text. Considering this findings, different hypothesis about word order is proposed. ChatGPT relies on word order to infer, but cannot support or negate the redundancy relations between word order lexical semantics.
  arXiv  Detail & Related papers  (2024-03-18T04:45:44Z)
• Word Order and World Knowledge [9.22384870426709]
  We study how word order affects the induction of world knowledge from raw text using language models. Specifically, in addition to the natural word order, we first respectively extract texts of six fixed word orders from five languages.
  arXiv  Detail & Related papers  (2024-03-01T08:13:48Z)
• A Cross-Linguistic Pressure for Uniform Information Density in Word Order [79.54362557462359]
  We use computational models to test whether real orders lead to greater information uniformity than counterfactual orders. Among SVO languages, real word orders consistently have greater uniformity than reverse word orders. Only linguistically implausible counterfactual orders consistently exceed the uniformity of real orders.
  arXiv  Detail & Related papers  (2023-06-06T14:52:15Z)
• Towards preserving word order importance through Forced Invalidation [80.33036864442182]
  We show that pre-trained language models are insensitive to word order. We propose Forced Invalidation to help preserve the importance of word order. Our experiments demonstrate that Forced Invalidation significantly improves the sensitivity of the models to word order.
  arXiv  Detail & Related papers  (2023-04-11T13:42:10Z)
• Transparency Helps Reveal When Language Models Learn Meaning [71.96920839263457]
  Our systematic experiments with synthetic data reveal that, with languages where all expressions have context-independent denotations, both autoregressive and masked language models learn to emulate semantic relations between expressions. Turning to natural language, our experiments with a specific phenomenon -- referential opacity -- add to the growing body of evidence that current language models do not well-represent natural language semantics.
  arXiv  Detail & Related papers  (2022-10-14T02:35:19Z)
• Word Order Does Matter (And Shuffled Language Models Know It) [9.990431777927421]
  Recent studies have shown that language models pretrained and/or fine-tuned on randomly permuted sentences exhibit competitive performance on GLUE. We investigate what position embeddings learned from shuffled text encode, showing that these models retain information pertaining to the original, naturalistic word order.
  arXiv  Detail & Related papers  (2022-03-21T14:10:15Z)
• Studying word order through iterative shuffling [14.530986799844873]
  We show that word order encodes meaning essential to performing NLP benchmark tasks. We use IBIS, a novel, efficient procedure that finds the ordering of a bag of words having the highest likelihood under a fixed language model. We discuss how shuffling inference procedures such as IBIS can benefit language modeling and constrained generation.
  arXiv  Detail & Related papers  (2021-09-10T13:27:06Z)
• Generalized Optimal Linear Orders [9.010643838773477]
  The sequential structure of language, and the order of words in a sentence specifically, plays a central role in human language processing. In designing computational models of language, the de facto approach is to present sentences to machines with the words ordered in the same order as in the original human-authored sentence. The very essence of this work is to question the implicit assumption that this is desirable and inject theoretical soundness into the consideration of word order in natural language processing.
  arXiv  Detail & Related papers  (2021-08-13T13:10:15Z)
• Investigating Cross-Linguistic Adjective Ordering Tendencies with a Latent-Variable Model [66.84264870118723]
  We present the first purely corpus-driven model of multi-lingual adjective ordering in the form of a latent-variable model. We provide strong converging evidence for the existence of universal, cross-linguistic, hierarchical adjective ordering tendencies.
  arXiv  Detail & Related papers  (2020-10-09T18:27:55Z)
• Speakers Fill Lexical Semantic Gaps with Context [65.08205006886591]
  We operationalise the lexical ambiguity of a word as the entropy of meanings it can take. We find significant correlations between our estimate of ambiguity and the number of synonyms a word has in WordNet. This suggests that, in the presence of ambiguity, speakers compensate by making contexts more informative.
  arXiv  Detail & Related papers  (2020-10-05T17:19:10Z)
• On the Importance of Word Order Information in Cross-lingual Sequence Labeling [80.65425412067464]
  Cross-lingual models that fit into the word order of the source language might fail to handle target languages. We investigate whether making models insensitive to the word order of the source language can improve the adaptation performance in target languages.
  arXiv  Detail & Related papers  (2020-01-30T03:35:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.