Recursive Tree Grammar Autoencoders

• URL: http://arxiv.org/abs/2012.02097v2
• Date: Tue, 15 Dec 2020 13:06:01 GMT
• Title: Recursive Tree Grammar Autoencoders
• Authors: Benjamin Paassen, Irena Koprinska, Kalina Yacef
• Abstract summary: We propose a novel autoencoder approach that encodes trees via a bottom-up grammar and decodes trees via a tree grammar. We show experimentally that our proposed method improves the autoencoding error, training time, and optimization score on four benchmark datasets.
• Score: 3.791857415239352
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Machine learning on tree data has been mostly focused on trees as input. Much less research has investigates trees as output, like in molecule optimization for drug discovery or hint generation for intelligent tutoring systems. In this work, we propose a novel autoencoder approach, called recursive tree grammar autoencoder (RTG-AE), which encodes trees via a bottom-up parser and decodes trees via a tree grammar, both controlled by neural networks that minimize the variational autoencoder loss. The resulting encoding and decoding functions can then be employed in subsequent tasks, such as optimization and time series prediction. RTG-AE combines variational autoencoders, grammatical knowledge, and recursive processing. Our key message is that this combination improves performance compared to only combining two of these three components. In particular, we show experimentally that our proposed method improves the autoencoding error, training time, and optimization score on four benchmark datasets compared to baselines from the literature.

Related papers

• Triple-Encoders: Representations That Fire Together, Wire Together [51.15206713482718]
  Contrastive Learning is a representation learning method that encodes relative distances between utterances into the embedding space via a bi-encoder. This study introduces triple-encoders, which efficiently compute distributed utterance mixtures from these independently encoded utterances. We find that triple-encoders lead to a substantial improvement over bi-encoders, and even to better zero-shot generalization than single-vector representation models.
  arXiv  Detail & Related papers  (2024-02-19T18:06:02Z)
• TreePiece: Faster Semantic Parsing via Tree Tokenization [2.1685554819849613]
  TreePiece tokenizes a parse tree into subtrees and generates one subtree per decoding step. On TopV2 benchmark, TreePiece shows 4.6 times faster decoding speed than standard AR.
  arXiv  Detail & Related papers  (2023-03-30T05:44:44Z)
• Structure-Unified M-Tree Coding Solver for MathWord Problem [57.825176412485504]
  In previous work, models designed by taking into account the properties of the binary tree structure of mathematical expressions at the output side have achieved better performance. In this paper, we propose the Structure-Unified M-Tree Coding Coding (S-UMCr), which applies a tree with any M branches (M-tree) to unify the output structures. Experimental results on the widely used MAWPS and Math23K datasets have demonstrated that SUMC-r not only outperforms several state-of-the-art models but also performs much better under low-resource conditions.
  arXiv  Detail & Related papers  (2022-10-22T12:20:36Z)
• Incorporating Constituent Syntax for Coreference Resolution [50.71868417008133]
  We propose a graph-based method to incorporate constituent syntactic structures. We also explore to utilise higher-order neighbourhood information to encode rich structures in constituent trees. Experiments on the English and Chinese portions of OntoNotes 5.0 benchmark show that our proposed model either beats a strong baseline or achieves new state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-02-22T07:40:42Z)
• Structural Optimization Makes Graph Classification Simpler and Better [5.770986723520119]
  We investigate the feasibility of improving graph classification performance while simplifying the model learning process. Inspired by progress in structural information assessment, we optimize the given data sample from graphs to encoding trees. We present an implementation of the scheme in a tree kernel and a convolutional network to perform graph classification.
  arXiv  Detail & Related papers  (2021-09-05T08:54:38Z)
• Autoencoders as Tools for Program Synthesis [0.43012765978447565]
  We introduce a variational autoencoder model for program synthesis of industry-grade programming languages. Our model incorporates the internal hierarchical structure of source codes and operates on parse trees.
  arXiv  Detail & Related papers  (2021-08-16T14:51:11Z)
• Recursive Top-Down Production for Sentence Generation with Latent Trees [77.56794870399288]
  We model the production property of context-free grammars for natural and synthetic languages. We present a dynamic programming algorithm that marginalises over latent binary tree structures with $N$ leaves. We also present experimental results on German-English translation on the Multi30k dataset.
  arXiv  Detail & Related papers  (2020-10-09T17:47:16Z)
• Transformer-Based Neural Text Generation with Syntactic Guidance [0.0]
  We study the problem of using (partial) constituency parse trees as syntactic guidance for controlled text generation. Our method first expands a partial template parse tree to a full-fledged parse tree tailored for the input source text. Our experiments in the controlled paraphrasing task show that our method outperforms SOTA models both semantically and syntactically.
  arXiv  Detail & Related papers  (2020-10-05T01:33:58Z)
• Tree Echo State Autoencoders with Grammars [3.7280152311394827]
  Non-vectorial and discrete nature of trees makes it challenging to construct functions with tree-formed output. Existing autoencoding approaches fail to take the specific grammatical structure of tree domains into account. We propose tree echo state autoencoders (TES-AE), which are guided by a tree grammar and can be trained within seconds by virtue of reservoir computing.
  arXiv  Detail & Related papers  (2020-04-19T18:04:33Z)
• Auto-Encoding Twin-Bottleneck Hashing [141.5378966676885]
  This paper proposes an efficient and adaptive code-driven graph. It is updated by decoding in the context of an auto-encoder. Experiments on benchmarked datasets clearly show the superiority of our framework over the state-of-the-art hashing methods.
  arXiv  Detail & Related papers  (2020-02-27T05:58:12Z)
• Learning Autoencoders with Relational Regularization [89.53065887608088]
  A new framework is proposed for learning autoencoders of data distributions. We minimize the discrepancy between the model and target distributions, with a emphrelational regularization We implement the framework with two scalable algorithms, making it applicable for both probabilistic and deterministic autoencoders.
  arXiv  Detail & Related papers  (2020-02-07T17:27:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.