Holographic Neural PCFG for Unsupervised Parsing

• URL: http://arxiv.org/abs/2607.08063v1
• Date: Thu, 09 Jul 2026 02:29:36 GMT
• Title: Holographic Neural PCFG for Unsupervised Parsing
• Abstract summary: Holographic Neural PCFG recasts PCFG rule scoring as algebra-symbol embeddings.<n>Hol-PCFG achieves state-of-the-art parsing performance in six languages.
• Score: 14.432298879095555
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Unsupervised constituency parsing aims to accurately induce latent tree structures from raw text alone. Recent neural parameterizations of PCFGs achieve strong performance in both supervised and unsupervised parsing, yet rely on high-capacity black-box networks for rule scoring -- as exemplified by the Neural PCFG family -- leaving rule probabilities without an interpretable mathematical form. In this paper, we propose Holographic Neural PCFG (Hol-PCFG), which recasts PCFG rule scoring as algebraic relation modeling among grammar-symbol embeddings. Hol-PCFG adapts Holographic Embeddings (Nickel et al., 2016), which scores knowledge-graph triples via circular correlation, to the left-child, right-child, and lexical-emission relations over torus-constrained embeddings, giving every rule probability a closed form that carries the intrinsic structure of grammar rules by construction. Hol-PCFG achieves state-of-the-art parsing performance in six languages while cutting rule-scoring parameters by 99.94% relative to the baseline model and training more stably. Additionally, we demonstrate that Hol-PCFG can parse Japanese directly from characters without any morphological segmentation, retaining nearly the same morpheme-level performance.

Related papers

• DPH Parser: A Bottom-Up Grammar-Driven Parser for Joint Constituency and Dependency Analysis [0.0]
  This paper presents a grammar-driven bottom-up unsupervized parsing framework inspired by Generalized Phrase Structure Grammar (GPSG) and Head-driven Phrase Structure Grammar (HPSG)<n>The system combines probabilistic POScorpora tagging, phrase projection, and weighted hypotheses to process realistic and partially noisy text input.<n>Although the current system does not yet approach the accuracy of modern neurals, the results demonstrate the feasibility of applying transparent rule-based bottom-up parsing to treebank data.
  arXiv  Detail & Related papers  (2026-09-05T12:58:58Z)
• Improving LLM Reasoning with Homophily-aware Structural and Semantic Text-Attributed Graph Compression [55.51959317490934]
  Large language models (LLMs) have demonstrated promising capabilities in Text-Attributed Graph (TAG) understanding.<n>We argue that graphs inherently contain rich structural and semantic information, and that their effective exploitation can unlock potential gains in LLMs reasoning performance.<n>We propose Homophily-aware Structural and Semantic Compression for LLMs (HS2C), a framework centered on exploiting graph homophily.
  arXiv  Detail & Related papers  (2026-01-13T03:35:18Z)
• Simple Hardware-Efficient PCFGs with Independent Left and Right Productions [77.12660133995362]
  This work introduces emphSimplePCFG, a simple PCFG formalism with independent left and right productions. As an unsupervised algorithm, our simple PCFG obtains an average F1 of 65.1 on the English PTB, and as a language model, it obtains a perplexity of 119.0, outperforming similarly-sized low-rank PCFGs.
  arXiv  Detail & Related papers  (2023-10-23T14:48:51Z)
• Stay on topic with Classifier-Free Guidance [57.28934343207042]
  We show that CFG can be used broadly as an inference-time technique in pure language modeling. We show that CFG improves the performance of Pythia, GPT-2 and LLaMA-family models across an array of tasks.
  arXiv  Detail & Related papers  (2023-06-30T17:07:02Z)
• Physics of Language Models: Part 1, Learning Hierarchical Language Structures [51.68385617116854]
  Transformer-based language models are effective but complex, and understanding their inner workings and reasoning mechanisms is a significant challenge.<n>We introduce a family of synthetic CFGs that produce hierarchical rules, capable of generating lengthy sentences.<n>We demonstrate that generative models like GPT can accurately learn and reason over CFG-defined hierarchies and generate sentences based on it.
  arXiv  Detail & Related papers  (2023-05-23T04:28:16Z)
• OrthoReg: Improving Graph-regularized MLPs via Orthogonality Regularization [66.30021126251725]
  Graph Neural Networks (GNNs) are currently dominating in modeling graphstructure data. Graph-regularized networks (GR-MLPs) implicitly inject the graph structure information into model weights, while their performance can hardly match that of GNNs in most tasks. We show that GR-MLPs suffer from dimensional collapse, a phenomenon in which the largest a few eigenvalues dominate the embedding space. We propose OrthoReg, a novel GR-MLP model to mitigate the dimensional collapse issue.
  arXiv  Detail & Related papers  (2023-01-31T21:20:48Z)
• Unsupervised Discontinuous Constituency Parsing with Mildly Context-Sensitive Grammars [14.256041558454786]
  We study grammar induction with mildly context-sensitive grammars for unsupervised discontinuous parsing. Our approach fixes the rule structure in advance and focuses on parameter learning with maximum likelihood. Experiments on German and Dutch show that our approach is able to induce linguistically meaningful trees with continuous and discontinuous structures.
  arXiv  Detail & Related papers  (2022-12-18T18:10:45Z)
• Neural Bi-Lexicalized PCFG Induction [22.728124473130876]
  We propose an approach to parameterize L-PCFGs without making implausible independence assumptions. Our approach directly models bilexical dependencies and meanwhile reduces both learning and representation complexities of L-PCFGs.
  arXiv  Detail & Related papers  (2021-05-31T15:00:03Z)
• PCFGs Can Do Better: Inducing Probabilistic Context-Free Grammars with Many Symbols [22.728124473130876]
  We present a new parameterization form of PCFGs based on tensor decomposition. We use neural parameterization for the new form to improve unsupervised parsing performance. We evaluate our model across ten languages and empirically demonstrate the effectiveness of using more symbols.
  arXiv  Detail & Related papers  (2021-04-28T12:25:27Z)
• An Empirical Study of Compound PCFGs [35.64371385720051]
  Compound context-free grammars (C-PCFGs) have recently established a new state of the art for unsupervised phrase-structure grammar induction. We analyze C-PCFGs on English treebanks and conduct a multilingual evaluation of C-PCFGs. The experimental results show that the best configurations of C-PCFGs, which are tuned on English, do not always generalize to morphology-rich languages.
  arXiv  Detail & Related papers  (2021-03-03T10:24:26Z)
• The Return of Lexical Dependencies: Neural Lexicalized PCFGs [103.41187595153652]
  We present novel neural models of lexicalized PCFGs which allow us to overcome sparsity problems. Experiments demonstrate that this unified framework results in stronger results on both representations than achieved when either formalism alone.
  arXiv  Detail & Related papers  (2020-07-29T22:12:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.