Randomized Deep Structured Prediction for Discourse-Level Processing

• URL: http://arxiv.org/abs/2101.10435v1
• Date: Mon, 25 Jan 2021 21:49:32 GMT
• Title: Randomized Deep Structured Prediction for Discourse-Level Processing
• Authors: Manuel Widmoser, Maria Leonor Pacheco, Jean Honorio, Dan Goldwasser
• Abstract summary: Expressive text encoders have been at the center of NLP models in recent work. We show that we can efficiently leverage deep structured prediction and expressive neural encoders for a set of tasks involving complicated argumentative structures.
• Score: 45.725437752821655
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Expressive text encoders such as RNNs and Transformer Networks have been at the center of NLP models in recent work. Most of the effort has focused on sentence-level tasks, capturing the dependencies between words in a single sentence, or pairs of sentences. However, certain tasks, such as argumentation mining, require accounting for longer texts and complicated structural dependencies between them. Deep structured prediction is a general framework to combine the complementary strengths of expressive neural encoders and structured inference for highly structured domains. Nevertheless, when the need arises to go beyond sentences, most work relies on combining the output scores of independently trained classifiers. One of the main reasons for this is that constrained inference comes at a high computational cost. In this paper, we explore the use of randomized inference to alleviate this concern and show that we can efficiently leverage deep structured prediction and expressive neural encoders for a set of tasks involving complicated argumentative structures.

Related papers

• Semantic Loss Functions for Neuro-Symbolic Structured Prediction [74.18322585177832]
  We discuss the semantic loss, which injects knowledge about such structure, defined symbolically, into training. It is agnostic to the arrangement of the symbols, and depends only on the semantics expressed thereby. It can be combined with both discriminative and generative neural models.
  arXiv  Detail & Related papers  (2024-05-12T22:18:25Z)
• Promptly Predicting Structures: The Return of Inference [31.442123334313035]
  We present a framework for constructing zero- and few-shot linguistic structure predictors. Our results show that enforcing consistency constructs not only structurally valid outputs, but also improves performance.
  arXiv  Detail & Related papers  (2024-01-12T20:08:39Z)
• Unsupervised Chunking with Hierarchical RNN [62.15060807493364]
  This paper introduces an unsupervised approach to chunking, a syntactic task that involves grouping words in a non-hierarchical manner. We present a two-layer Hierarchical Recurrent Neural Network (HRNN) designed to model word-to-chunk and chunk-to-sentence compositions. Experiments on the CoNLL-2000 dataset reveal a notable improvement over existing unsupervised methods, enhancing phrase F1 score by up to 6 percentage points.
  arXiv  Detail & Related papers  (2023-09-10T02:55:12Z)
• Extracting Text Representations for Terms and Phrases in Technical Domains [9.27244202193623]
  We propose a fully unsupervised approach to text encoding that consists of training small character-based models with the objective of reconstructing large pre-trained embedding matrices. Models trained with this approach can not only match the quality of sentence encoders in technical domains, but are 5 times smaller and up to 10 times faster.
  arXiv  Detail & Related papers  (2023-05-25T08:59:36Z)
• Topic-driven Distant Supervision Framework for Macro-level Discourse Parsing [72.14449502499535]
  The task of analyzing the internal rhetorical structure of texts is a challenging problem in natural language processing. Despite the recent advances in neural models, the lack of large-scale, high-quality corpora for training remains a major obstacle. Recent studies have attempted to overcome this limitation by using distant supervision.
  arXiv  Detail & Related papers  (2023-05-23T07:13:51Z)
• Equivariant Transduction through Invariant Alignment [71.45263447328374]
  We introduce a novel group-equivariant architecture that incorporates a group-in hard alignment mechanism. We find that our network's structure allows it to develop stronger equivariant properties than existing group-equivariant approaches. We additionally find that it outperforms previous group-equivariant networks empirically on the SCAN task.
  arXiv  Detail & Related papers  (2022-09-22T11:19:45Z)
• Nested Named Entity Recognition as Holistic Structure Parsing [92.8397338250383]
  This work models the full nested NEs in a sentence as a holistic structure, then we propose a holistic structure parsing algorithm to disclose the entire NEs once for all. Experiments show that our model yields promising results on widely-used benchmarks which approach or even achieve state-of-the-art.
  arXiv  Detail & Related papers  (2022-04-17T12:48:20Z)
• Neuro-Symbolic Constraint Programming for Structured Prediction [32.427665902031436]
  We propose Nester, a method for injecting neural networks into constrained structured predictors. Nester takes advantage of the features of its two components: the neural network learns complex representations from low-level data. An empirical evaluation on handwritten equation recognition shows that Nester achieves better performance than both the neural network and the constrained structured predictor.
  arXiv  Detail & Related papers  (2021-03-31T17:31:33Z)
• Discontinuous Constituent Parsing with Pointer Networks [0.34376560669160383]
  discontinuous constituent trees are crucial for representing all grammatical phenomena of languages such as German. Recent advances in dependency parsing have shown that Pointer Networks excel in efficiently parsing syntactic relations between words in a sentence. We propose a novel neural network architecture that is able to generate the most accurate discontinuous constituent representations.
  arXiv  Detail & Related papers  (2020-02-05T15:12:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.