A Hybrid Approach to Dependency Parsing: Combining Rules and Morphology with Deep Learning

• URL: http://arxiv.org/abs/2002.10116v1
• Date: Mon, 24 Feb 2020 08:34:33 GMT
• Title: A Hybrid Approach to Dependency Parsing: Combining Rules and Morphology with Deep Learning
• Authors: \c{S}aziye Bet\"ul \"Ozate\c{s} (1), Arzucan \"Ozg\"ur (1), Tunga G\"ung\"or (1), Balk{\i}z \"Ozt\"urk (2) ((1) Department of Computer Engineering, Bo\u{g}azi\c{c}i University, (2) Department of Linguistics, Bo\u{g}azi\c{c}i University)
• Abstract summary: We propose two approaches to dependency parsing especially for languages with restricted amount of training data. Our first approach combines a state-of-the-art deep learning-based with a rule-based approach and the second one incorporates morphological information into the network. The proposed methods are developed for Turkish, but can be adapted to other languages as well.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Fully data-driven, deep learning-based models are usually designed as language-independent and have been shown to be successful for many natural language processing tasks. However, when the studied language is low-resourced and the amount of training data is insufficient, these models can benefit from the integration of natural language grammar-based information. We propose two approaches to dependency parsing especially for languages with restricted amount of training data. Our first approach combines a state-of-the-art deep learning-based parser with a rule-based approach and the second one incorporates morphological information into the parser. In the rule-based approach, the parsing decisions made by the rules are encoded and concatenated with the vector representations of the input words as additional information to the deep network. The morphology-based approach proposes different methods to include the morphological structure of words into the parser network. Experiments are conducted on the IMST-UD Treebank and the results suggest that integration of explicit knowledge about the target language to a neural parser through a rule-based parsing system and morphological analysis leads to more accurate annotations and hence, increases the parsing performance in terms of attachment scores. The proposed methods are developed for Turkish, but can be adapted to other languages as well.

Related papers

• Learning Phonotactics from Linguistic Informants [54.086544221761486]
  Our model iteratively selects or synthesizes a data-point according to one of a range of information-theoretic policies. We find that the information-theoretic policies that our model uses to select items to query the informant achieve sample efficiency comparable to, or greater than, fully supervised approaches.
  arXiv  Detail & Related papers  (2024-05-08T00:18:56Z)
• Transparency at the Source: Evaluating and Interpreting Language Models With Access to the True Distribution [4.01799362940916]
  We present a setup for training, evaluating and interpreting neural language models, that uses artificial, language-like data. The data is generated using a massive probabilistic grammar, that is itself derived from a large natural language corpus. With access to the underlying true source, our results show striking differences and outcomes in learning dynamics between different classes of words.
  arXiv  Detail & Related papers  (2023-10-23T12:03:01Z)
• Soft Language Clustering for Multilingual Model Pre-training [57.18058739931463]
  We propose XLM-P, which contextually retrieves prompts as flexible guidance for encoding instances conditionally. Our XLM-P enables (1) lightweight modeling of language-invariant and language-specific knowledge across languages, and (2) easy integration with other multilingual pre-training methods.
  arXiv  Detail & Related papers  (2023-06-13T08:08:08Z)
• On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codex [48.588772371355816]
  This paper presents the first empirical study on the adversarial robustness of a large prompt-based language model of code, codex. Our results demonstrate that the state-of-the-art (SOTA) code-language models are vulnerable to carefully crafted adversarial examples.
  arXiv  Detail & Related papers  (2023-01-30T13:21:00Z)
• Evaluating the Morphosyntactic Well-formedness of Generated Texts [88.20502652494521]
  We propose L'AMBRE -- a metric to evaluate the morphosyntactic well-formedness of text. We show the effectiveness of our metric on the task of machine translation through a diachronic study of systems translating into morphologically-rich languages.
  arXiv  Detail & Related papers  (2021-03-30T18:02:58Z)
• Multilingual Neural RST Discourse Parsing [24.986030179701405]
  We investigate two approaches to establish a neural, cross-lingual discourse via multilingual vector representations and segment-level translation. Experiment results show that both methods are effective even with limited training data, and achieve state-of-the-art performance on cross-lingual, document-level discourse parsing.
  arXiv  Detail & Related papers  (2020-12-03T05:03:38Z)
• SLM: Learning a Discourse Language Representation with Sentence Unshuffling [53.42814722621715]
  We introduce Sentence-level Language Modeling, a new pre-training objective for learning a discourse language representation. We show that this feature of our model improves the performance of the original BERT by large margins.
  arXiv  Detail & Related papers  (2020-10-30T13:33:41Z)
• Grounded Compositional Outputs for Adaptive Language Modeling [59.02706635250856]
  A language model's vocabulary$-$typically selected before training and permanently fixed later$-$affects its size. We propose a fully compositional output embedding layer for language models. To our knowledge, the result is the first word-level language model with a size that does not depend on the training vocabulary.
  arXiv  Detail & Related papers  (2020-09-24T07:21:14Z)
• Systematic Generalization on gSCAN with Language Conditioned Embedding [19.39687991647301]
  Systematic Generalization refers to a learning algorithm's ability to extrapolate learned behavior to unseen situations. We propose a novel method that learns objects' contextualized embeddings with dynamic message passing conditioned on the input natural language.
  arXiv  Detail & Related papers  (2020-09-11T17:35:05Z)
• Learning Spoken Language Representations with Neural Lattice Language Modeling [39.50831917042577]
  We propose a framework that trains neural lattice language models to provide contextualized representations for spoken language understanding tasks. The proposed two-stage pre-training approach reduces the demands of speech data and has better efficiency.
  arXiv  Detail & Related papers  (2020-07-06T10:38:03Z)
• Multilingual Chart-based Constituency Parse Extraction from Pre-trained Language Models [21.2879567125422]
  We propose a novel method for extracting complete (binary) parses from pre-trained language models. By applying our method on multilingual PLMs, it becomes possible to induce non-trivial parses for sentences from nine languages.
  arXiv  Detail & Related papers  (2020-04-08T05:42:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.