Is POS Tagging Necessary or Even Helpful for Neural Dependency Parsing?

• URL: http://arxiv.org/abs/2003.03204v2
• Date: Fri, 14 Aug 2020 06:41:31 GMT
• Title: Is POS Tagging Necessary or Even Helpful for Neural Dependency Parsing?
• Authors: Houquan Zhou, Yu Zhang, Zhenghua Li, Min Zhang
• Abstract summary: We show that POS tagging can still significantly improve parsing performance when using the Stack joint framework. Considering that it is much cheaper to annotate POS tags than parse trees, we also investigate the utilization of large-scale heterogeneous POS tag data.
• Score: 22.93722845643562
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In the pre deep learning era, part-of-speech tags have been considered as indispensable ingredients for feature engineering in dependency parsing. But quite a few works focus on joint tagging and parsing models to avoid error propagation. In contrast, recent studies suggest that POS tagging becomes much less important or even useless for neural parsing, especially when using character-based word representations. Yet there are not enough investigations focusing on this issue, both empirically and linguistically. To answer this, we design and compare three typical multi-task learning framework, i.e., Share-Loose, Share-Tight, and Stack, for joint tagging and parsing based on the state-of-the-art biaffine parser. Considering that it is much cheaper to annotate POS tags than parse trees, we also investigate the utilization of large-scale heterogeneous POS tag data. We conduct experiments on both English and Chinese datasets, and the results clearly show that POS tagging (both homogeneous and heterogeneous) can still significantly improve parsing performance when using the Stack joint framework. We conduct detailed analysis and gain more insights from the linguistic aspect.

Related papers

• Urdu Dependency Parsing and Treebank Development: A Syntactic and Morphological Perspective [0.0]
  We use dependency parsing to analyze news articles in Urdu. We achieve a best-labeled accuracy (LA) of 70% and an unlabeled attachment score (UAS) of 84%.
  arXiv  Detail & Related papers  (2024-06-13T19:30:32Z)
• Towards Unsupervised Recognition of Token-level Semantic Differences in Related Documents [61.63208012250885]
  We formulate recognizing semantic differences as a token-level regression task. We study three unsupervised approaches that rely on a masked language model. Our results show that an approach based on word alignment and sentence-level contrastive learning has a robust correlation to gold labels.
  arXiv  Detail & Related papers  (2023-05-22T17:58:04Z)
• Parsing linearizations appreciate PoS tags - but some are fussy about errors [12.024457689086008]
  PoS tags, once taken for granted as a useful resource for syntactic parsing, have become more situational with the popularization of deep learning. Recent work on the impact of PoS tags on graph- and transition-based labelings suggests that they are only useful when tagging accuracy is high, or in low-resource scenarios. We undertake a study and uncover some trends. Among them, PoS tags are generally more useful for sequence labelings than for other paradigms, but the impact of their accuracy is highly encoding-dependent.
  arXiv  Detail & Related papers  (2022-10-27T07:15:36Z)
• Masked Part-Of-Speech Model: Does Modeling Long Context Help Unsupervised POS-tagging? [94.68962249604749]
  We propose a Masked Part-of-Speech Model (MPoSM) to facilitate flexible dependency modeling. MPoSM can model arbitrary tag dependency and perform POS induction through the objective of masked POS reconstruction. We achieve competitive results on both the English Penn WSJ dataset and the universal treebank containing 10 diverse languages.
  arXiv  Detail & Related papers  (2022-06-30T01:43:05Z)
• On The Ingredients of an Effective Zero-shot Semantic Parser [95.01623036661468]
  We analyze zero-shot learning by paraphrasing training examples of canonical utterances and programs from a grammar. We propose bridging these gaps using improved grammars, stronger paraphrasers, and efficient learning methods. Our model achieves strong performance on two semantic parsing benchmarks (Scholar, Geo) with zero labeled data.
  arXiv  Detail & Related papers  (2021-10-15T21:41:16Z)
• Contextualized Semantic Distance between Highly Overlapped Texts [85.1541170468617]
  Overlapping frequently occurs in paired texts in natural language processing tasks like text editing and semantic similarity evaluation. This paper aims to address the issue with a mask-and-predict strategy. We take the words in the longest common sequence as neighboring words and use masked language modeling (MLM) to predict the distributions on their positions. Experiments on Semantic Textual Similarity show NDD to be more sensitive to various semantic differences, especially on highly overlapped paired texts.
  arXiv  Detail & Related papers  (2021-10-04T03:59:15Z)
• Pareto Probing: Trading Off Accuracy for Complexity [87.09294772742737]
  We argue for a probe metric that reflects the fundamental trade-off between probe complexity and performance. Our experiments with dependency parsing reveal a wide gap in syntactic knowledge between contextual and non-contextual representations.
  arXiv  Detail & Related papers  (2020-10-05T17:27:31Z)
• A Survey of Unsupervised Dependency Parsing [62.16714720135358]
  Unsupervised dependency parsing aims to learn a dependency from sentences that have no annotation of their correct parse trees. Despite its difficulty, unsupervised parsing is an interesting research direction because of its capability of utilizing almost unlimited unannotated text data.
  arXiv  Detail & Related papers  (2020-10-04T10:51:22Z)
• Reliable Part-of-Speech Tagging of Historical Corpora through Set-Valued Prediction [21.67895423776014]
  We consider POS tagging within the framework of set-valued prediction. We find that extending state-of-the-art POS taggers to set-valued prediction yields more precise and robust taggings.
  arXiv  Detail & Related papers  (2020-08-04T07:21:36Z)
• A Cross-Genre Ensemble Approach to Robust Reddit Part of Speech Tagging [10.609715843964263]
  We study how a state-of-the-art tagging model trained on different genres performs on Web content from unfiltered Reddit forum discussions. Our results show that even small amounts of in-domain data can outperform the contribution of data from other Web domains.
  arXiv  Detail & Related papers  (2020-04-29T16:36:38Z)
• Machine Learning Approaches for Amharic Parts-of-speech Tagging [0.0]
  Performance of the current POS taggers in Amharic is not as good as that of the contemporary POS taggers available for English and other European languages. The aim of this work is to improve POS tagging performance for the Amharic language, which was never above 91%.
  arXiv  Detail & Related papers  (2020-01-10T06:40:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.