An Empirical Study of Compound PCFGs

• URL: http://arxiv.org/abs/2103.02298v2
• Date: Sat, 21 Oct 2023 17:26:25 GMT
• Title: An Empirical Study of Compound PCFGs
• Authors: Yanpeng Zhao, Ivan Titov
• Abstract summary: Compound context-free grammars (C-PCFGs) have recently established a new state of the art for unsupervised phrase-structure grammar induction. We analyze C-PCFGs on English treebanks and conduct a multilingual evaluation of C-PCFGs. The experimental results show that the best configurations of C-PCFGs, which are tuned on English, do not always generalize to morphology-rich languages.
• Score: 35.64371385720051
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Compound probabilistic context-free grammars (C-PCFGs) have recently established a new state of the art for unsupervised phrase-structure grammar induction. However, due to the high space and time complexities of chart-based representation and inference, it is difficult to investigate C-PCFGs comprehensively. In this work, we rely on a fast implementation of C-PCFGs to conduct an evaluation complementary to that of~\citet{kim-etal-2019-compound}. We start by analyzing and ablating C-PCFGs on English treebanks. Our findings suggest that (1) C-PCFGs are data-efficient and can generalize to unseen sentence/constituent lengths; and (2) C-PCFGs make the best use of sentence-level information in generating preterminal rule probabilities. We further conduct a multilingual evaluation of C-PCFGs. The experimental results show that the best configurations of C-PCFGs, which are tuned on English, do not always generalize to morphology-rich languages.

Related papers

• Nearest Neighbor CCP-Based Molecular Sequence Analysis [4.199844472131922]
  Correlated Clustering and Projection (CCP) has been proposed as an effective method for biological sequencing data. We present a Nearest Neighbor Correlated Clustering and Projection (CCP-NN)-based technique for efficiently preprocessing molecular sequence data. Our findings show that CCP-NN considerably improves classification task accuracy as well as significantly outperforms CCP in terms of computational runtime.
  arXiv  Detail & Related papers  (2024-09-07T22:06:00Z)
• Simple Hardware-Efficient PCFGs with Independent Left and Right Productions [77.12660133995362]
  This work introduces emphSimplePCFG, a simple PCFG formalism with independent left and right productions. As an unsupervised algorithm, our simple PCFG obtains an average F1 of 65.1 on the English PTB, and as a language model, it obtains a perplexity of 119.0, outperforming similarly-sized low-rank PCFGs.
  arXiv  Detail & Related papers  (2023-10-23T14:48:51Z)
• Convergence and Diversity in the Control Hierarchy [134.09048604473793]
  We show that four formalisms are not only weakly equivalent but equivalent in a stricter sense that we call d-weak equivalence. We make precise the intuition that a CFG controlling a CFG is a TAG, a PDA controlling a PDA is an embedded PDA, and a PDA controlling a CFG is a LIG. The fourth member of this family, a CFG controlling a PDA, does not correspond to any formalism we know of, so we invent one and call it a Pushdown Adjoining Automaton.
  arXiv  Detail & Related papers  (2023-06-06T12:30:29Z)
• Physics of Language Models: Part 1, Learning Hierarchical Language Structures [51.68385617116854]
  Transformer-based language models are effective but complex, and understanding their inner workings is a significant challenge. We introduce a family of synthetic CFGs that produce hierarchical rules, capable of generating lengthy sentences. We demonstrate that generative models like GPT can accurately learn this CFG language and generate sentences based on it.
  arXiv  Detail & Related papers  (2023-05-23T04:28:16Z)
• Re-evaluating the Need for Multimodal Signals in Unsupervised Grammar Induction [119.07444667700473]
  Recent work has shown that multimodal training inputs can improve grammar induction. We design a stronger text-only baseline, which we refer to as LC-PCFG. LC-PCFG provides an up to 17% relative improvement in Corpus-F1 compared to state-of-the-art multimodal grammar induction methods.
  arXiv  Detail & Related papers  (2022-12-20T18:59:50Z)
• Initialisation and Grammar Design in Grammar-Guided Evolutionary Computation [0.0]
  We show that genetic programming (CFG-GP) is less sensitive to initialisation and grammar design than random search and GE. We also demonstrate that observed cases of poor performance by CFG-GP are managed through simple adjustment of tuning parameters.
  arXiv  Detail & Related papers  (2022-04-15T10:15:40Z)
• Compositional Generalization Requires Compositional Parsers [69.77216620997305]
  We compare sequence-to-sequence models and models guided by compositional principles on the recent COGS corpus. We show structural generalization is a key measure of compositional generalization and requires models that are aware of complex structure.
  arXiv  Detail & Related papers  (2022-02-24T07:36:35Z)
• Neural Bi-Lexicalized PCFG Induction [22.728124473130876]
  We propose an approach to parameterize L-PCFGs without making implausible independence assumptions. Our approach directly models bilexical dependencies and meanwhile reduces both learning and representation complexities of L-PCFGs.
  arXiv  Detail & Related papers  (2021-05-31T15:00:03Z)
• PCFGs Can Do Better: Inducing Probabilistic Context-Free Grammars with Many Symbols [22.728124473130876]
  We present a new parameterization form of PCFGs based on tensor decomposition. We use neural parameterization for the new form to improve unsupervised parsing performance. We evaluate our model across ten languages and empirically demonstrate the effectiveness of using more symbols.
  arXiv  Detail & Related papers  (2021-04-28T12:25:27Z)
• Discriminative Learning for Probabilistic Context-Free Grammars based on Generalized H-Criterion [0.0]
  We present a family of discriminative learning algorithms for Probabilistic Context-Free Grammars (PCFGs) based on a generalization of criterion-H.
  arXiv  Detail & Related papers  (2021-03-15T19:07:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.