Average shortest-path length in word-adjacency networks: Chinese versus English

• URL: http://arxiv.org/abs/2601.06361v1
• Date: Sat, 10 Jan 2026 00:12:37 GMT
• Title: Average shortest-path length in word-adjacency networks: Chinese versus English
• Authors: Jakub Dec, Michał Dolina, Stanisław Drożdż, Jarosław Kwapień, Jin Liu, Tomasz Stanisz,
• Abstract summary: We analyze word-adjacency networks constructed from Chinese and English literary works written in different periods.<n>We show that punctuation marks behave like words in a Zipfian analysis and, if considered together with regular words, can improve authorship attribution.
• Score: 2.389158198260747
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Complex networks provide powerful tools for analyzing and understanding the intricate structures present in various systems, including natural language. Here, we analyze topology of growing word-adjacency networks constructed from Chinese and English literary works written in different periods. Unconventionally, instead of considering dictionary words only, we also include punctuation marks as if they were ordinary words. Our approach is based on two arguments: (1) punctuation carries genuine information related to emotional state, allows for logical grouping of content, provides a pause in reading, and facilitates understanding by avoiding ambiguity, and (2) our previous works have shown that punctuation marks behave like words in a Zipfian analysis and, if considered together with regular words, can improve authorship attribution in stylometric studies. We focus on a functional dependence of the average shortest path length $L(N)$ on a network size $N$ for different epochs and individual novels in their original language as well as for translations of selected novels into the other language. We approximate the empirical results with a growing network model and obtain satisfactory agreement between the two. We also observe that $L(N)$ behaves asymptotically similar for both languages if punctuation marks are included but becomes sizably larger for Chinese if punctuation marks are neglected.

Related papers

• False Friends Are Not Foes: Investigating Vocabulary Overlap in Multilingual Language Models [53.01170039144264]
  Subword tokenizers trained on multilingual corpora naturally produce overlapping tokens across languages.<n>Does token overlap facilitate cross-lingual transfer or instead introduce interference between languages?<n>We find that models with overlap outperform models with disjoint vocabularies.
  arXiv  Detail & Related papers  (2025-09-23T07:47:54Z)
• Tokens with Meaning: A Hybrid Tokenization Approach for NLP [0.2826977330147589]
  Tokenization plays a pivotal role in natural language processing (NLP)<n>We introduce a hybrid tokenization framework that combines rule-based morphological analysis with statistical subword segmentation.<n>The method uses phono normalization, root-affix, and a novel algorithm that balances morpheme preservation with vocabulary efficiency.
  arXiv  Detail & Related papers  (2025-08-19T22:17:42Z)
• Parsing Through Boundaries in Chinese Word Segmentation [5.144001661743487]
  Unlike English, Chinese lacks explicit word boundaries, making segmentation both necessary and inherently ambiguous.<n>This study highlights the intricate relationship between word segmentation and syntactic parsing, providing a clearer understanding of how different segmentation strategies shape dependency structures in Chinese.
  arXiv  Detail & Related papers  (2025-03-29T14:24:02Z)
• Entropy and type-token ratio in gigaword corpora [0.0]
  lexical diversity is characterized in terms of the type-token ratio and the word entropy.<n>We investigate both diversity metrics in six massive linguistic datasets in English, Spanish, and Turkish.<n>We unveil an empirical functional relation between entropy and type-token ratio of texts of a given corpus and language.
  arXiv  Detail & Related papers  (2024-11-15T14:40:59Z)
• A General and Flexible Multi-concept Parsing Framework for Multilingual Semantic Matching [60.51839859852572]
  We propose to resolve the text into multi concepts for multilingual semantic matching to liberate the model from the reliance on NER models. We conduct comprehensive experiments on English datasets QQP and MRPC, and Chinese dataset Medical-SM.
  arXiv  Detail & Related papers  (2024-03-05T13:55:16Z)
• Syntactic Language Change in English and German: Metrics, Parsers, and Convergences [56.47832275431858]
  The current paper looks at diachronic trends in syntactic language change in both English and German, using corpora of parliamentary debates from the last c. 160 years. We base our observations on five dependencys, including the widely used Stanford Core as well as 4 newer alternatives. We show that changes in syntactic measures seem to be more frequent at the tails of sentence length distributions.
  arXiv  Detail & Related papers  (2024-02-18T11:46:16Z)
• Complex systems approach to natural language [0.0]
  Review summarizes the main methodological concepts used in studying natural language from the perspective of complexity science. Three main complexity-related research trends in quantitative linguistics are covered.
  arXiv  Detail & Related papers  (2024-01-05T12:01:26Z)
• Universal versus system-specific features of punctuation usage patterns in~major Western~languages [0.0]
  In written texts punctuation can be considered one of its manifestations. This study is based on a large corpus of world-famous and representative literary texts in seven major Western languages.
  arXiv  Detail & Related papers  (2022-12-21T16:52:10Z)
• More Than Words: Collocation Tokenization for Latent Dirichlet Allocation Models [71.42030830910227]
  We propose a new metric for measuring the clustering quality in settings where the models differ. We show that topics trained with merged tokens result in topic keys that are clearer, more coherent, and more effective at distinguishing topics than those unmerged models.
  arXiv  Detail & Related papers  (2021-08-24T14:08:19Z)
• An In-depth Study on Internal Structure of Chinese Words [34.864343591706984]
  This work proposes to model the deep internal structures of Chinese words as dependency trees with 11 labels for distinguishing syntactic relationships. We manually annotate a word-internal structure treebank (WIST) consisting of over 30K multi-char words from Chinese Penn Treebank. We present detailed and interesting analysis on WIST to reveal insights on Chinese word formation.
  arXiv  Detail & Related papers  (2021-06-01T09:09:51Z)
• Speakers Fill Lexical Semantic Gaps with Context [65.08205006886591]
  We operationalise the lexical ambiguity of a word as the entropy of meanings it can take. We find significant correlations between our estimate of ambiguity and the number of synonyms a word has in WordNet. This suggests that, in the presence of ambiguity, speakers compensate by making contexts more informative.
  arXiv  Detail & Related papers  (2020-10-05T17:19:10Z)
• On the Language Neutrality of Pre-trained Multilingual Representations [70.93503607755055]
  We investigate the language-neutrality of multilingual contextual embeddings directly and with respect to lexical semantics. Our results show that contextual embeddings are more language-neutral and, in general, more informative than aligned static word-type embeddings. We show how to reach state-of-the-art accuracy on language identification and match the performance of statistical methods for word alignment of parallel sentences.
  arXiv  Detail & Related papers  (2020-04-09T19:50:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.