Rethinking the Relationship between the Power Law and Hierarchical Structures

• URL: http://arxiv.org/abs/2505.04984v1
• Date: Thu, 08 May 2025 06:41:46 GMT
• Title: Rethinking the Relationship between the Power Law and Hierarchical Structures
• Authors: Kai Nakaishi, Ryo Yoshida, Kohei Kajikawa, Koji Hukushima, Yohei Oseki,
• Abstract summary: This study examines the validity of the argument for syntactic structures using English corpora.<n>Using English corpora, we analyze the mutual information, deviations from probabilistic context-free grammars, and other properties in parse trees.<n>Our results indicate that the assumptions do not hold for syntactic structures and that it is difficult to apply the proposed argument to child languages and animal signals.
• Score: 3.8063235092267993
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Statistical analysis of corpora provides an approach to quantitatively investigate natural languages. This approach has revealed that several power laws consistently emerge across different corpora and languages, suggesting the universal principles underlying languages. Particularly, the power-law decay of correlation has been interpreted as evidence for underlying hierarchical structures in syntax, semantics, and discourse. This perspective has also been extended to child languages and animal signals. However, the argument supporting this interpretation has not been empirically tested. To address this problem, this study examines the validity of the argument for syntactic structures. Specifically, we test whether the statistical properties of parse trees align with the implicit assumptions in the argument. Using English corpora, we analyze the mutual information, deviations from probabilistic context-free grammars (PCFGs), and other properties in parse trees, as well as in the PCFG that approximates these trees. Our results indicate that the assumptions do not hold for syntactic structures and that it is difficult to apply the proposed argument to child languages and animal signals, highlighting the need to reconsider the relationship between the power law and hierarchical structures.

Related papers

• Counting trees: A treebank-driven exploration of syntactic variation in speech and writing across languages [0.0]
  We define syntactic structures as delexicalized dependency (sub)trees and extract them from spoken and written Universal Dependencies treebanks.<n>For each corpus, we analyze the size, diversity, and distribution of syntactic inventories, their overlap across modalities, and the structures most characteristic of speech.<n>Results show that, across both languages, spoken corpora contain fewer and less diverse syntactic structures than their written counterparts.
  arXiv  Detail & Related papers  (2025-05-28T18:43:26Z)
• SPAWNing Structural Priming Predictions from a Cognitively Motivated Parser [20.83447226847776]
  We use SPAWN to generate priming predictions from two theories about the structure of English relative clauses. We found that while the some of the predictions from the Participial-Phase theory aligned with human behavior, none of the predictions from the the Whiz-Deletion theory did.
  arXiv  Detail & Related papers  (2024-03-11T22:58:58Z)
• Zero-shot Causal Graph Extrapolation from Text via LLMs [50.596179963913045]
  We evaluate the ability of large language models (LLMs) to infer causal relations from natural language. LLMs show competitive performance in a benchmark of pairwise relations without needing (explicit) training samples. We extend our approach to extrapolating causal graphs through iterated pairwise queries.
  arXiv  Detail & Related papers  (2023-12-22T13:14:38Z)
• How Well Do Text Embedding Models Understand Syntax? [50.440590035493074]
  The ability of text embedding models to generalize across a wide range of syntactic contexts remains under-explored. Our findings reveal that existing text embedding models have not sufficiently addressed these syntactic understanding challenges. We propose strategies to augment the generalization ability of text embedding models in diverse syntactic scenarios.
  arXiv  Detail & Related papers  (2023-11-14T08:51:00Z)
• Modeling Hierarchical Reasoning Chains by Linking Discourse Units and Key Phrases for Reading Comprehension [80.99865844249106]
  We propose a holistic graph network (HGN) which deals with context at both discourse level and word level, as the basis for logical reasoning. Specifically, node-level and type-level relations, which can be interpreted as bridges in the reasoning process, are modeled by a hierarchical interaction mechanism.
  arXiv  Detail & Related papers  (2023-06-21T07:34:27Z)
• Structural Ambiguity and its Disambiguation in Language Model Based Parsers: the Case of Dutch Clause Relativization [2.9950872478176627]
  We study how the presence of a prior sentence can resolve relative clause ambiguities. Results show that a neurosymbolic, based on proof nets, is more open to data bias correction than an approach based on universal dependencies.
  arXiv  Detail & Related papers  (2023-05-24T09:04:18Z)
• Natural Language Decompositions of Implicit Content Enable Better Text Representations [52.992875653864076]
  We introduce a method for the analysis of text that takes implicitly communicated content explicitly into account.<n>We use a large language model to produce sets of propositions that are inferentially related to the text that has been observed.<n>Our results suggest that modeling the meanings behind observed language, rather than the literal text alone, is a valuable direction for NLP.
  arXiv  Detail & Related papers  (2023-05-23T23:45:20Z)
• False perspectives on human language: why statistics needs linguistics [0.8699677835130408]
  We show that statistical measures can be defined on the basis of either structural or non-structural models. Only models of surprisal that reflect syntactic structure are able to account for language regularities.
  arXiv  Detail & Related papers  (2023-02-17T11:40:32Z)
• Exploring Discourse Structures for Argument Impact Classification [48.909640432326654]
  This paper empirically shows that the discourse relations between two arguments along the context path are essential factors for identifying the persuasive power of an argument. We propose DisCOC to inject and fuse the sentence-level structural information with contextualized features derived from large-scale language models.
  arXiv  Detail & Related papers  (2021-06-02T06:49:19Z)
• Linguistic dependencies and statistical dependence [76.89273585568084]
  We use pretrained language models to estimate probabilities of words in context. We find that maximum-CPMI trees correspond to linguistic dependencies more often than trees extracted from non-contextual PMI estimate.
  arXiv  Detail & Related papers  (2021-04-18T02:43:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.