Learning Syntactic and Dynamic Selective Encoding for Document
Summarization
- URL: http://arxiv.org/abs/2003.11173v1
- Date: Wed, 25 Mar 2020 01:29:38 GMT
- Title: Learning Syntactic and Dynamic Selective Encoding for Document
Summarization
- Authors: Haiyang Xu, Yahao He, Kun Han, Junwen Chen and Xiangang Li
- Abstract summary: We propose a novel neural architecture for document summarization.
We incorporate syntactic information such as constituency parsing trees into the encoding sequence.
We propose a dynamic gate network to select the salient information based on the context of the decoder state.
- Score: 17.666036645395845
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Text summarization aims to generate a headline or a short summary consisting
of the major information of the source text. Recent studies employ the
sequence-to-sequence framework to encode the input with a neural network and
generate abstractive summary. However, most studies feed the encoder with the
semantic word embedding but ignore the syntactic information of the text.
Further, although previous studies proposed the selective gate to control the
information flow from the encoder to the decoder, it is static during the
decoding and cannot differentiate the information based on the decoder states.
In this paper, we propose a novel neural architecture for document
summarization. Our approach has the following contributions: first, we
incorporate syntactic information such as constituency parsing trees into the
encoding sequence to learn both the semantic and syntactic information from the
document, resulting in more accurate summary; second, we propose a dynamic gate
network to select the salient information based on the context of the decoder
state, which is essential to document summarization. The proposed model has
been evaluated on CNN/Daily Mail summarization datasets and the experimental
results show that the proposed approach outperforms baseline approaches.
Related papers
- Neural Sequence-to-Sequence Modeling with Attention by Leveraging Deep Learning Architectures for Enhanced Contextual Understanding in Abstractive Text Summarization [0.0]
This paper presents a novel framework for abstractive TS of single documents.
It integrates three dominant aspects: structure, semantic, and neural-based approaches.
Results indicate significant improvements in handling rare and OOV words.
arXiv Detail & Related papers (2024-04-08T18:33:59Z) - Tram: A Token-level Retrieval-augmented Mechanism for Source Code Summarization [76.57699934689468]
We propose a fine-grained Token-level retrieval-augmented mechanism (Tram) on the decoder side to enhance the performance of neural models.
To overcome the challenge of token-level retrieval in capturing contextual code semantics, we also propose integrating code semantics into individual summary tokens.
arXiv Detail & Related papers (2023-05-18T16:02:04Z) - Multi-Document Scientific Summarization from a Knowledge Graph-Centric
View [9.579482432715261]
We present KGSum, an MDSS model centred on knowledge graphs during both the encoding and decoding process.
Specifically, in the encoding process, two graph-based modules are proposed to incorporate knowledge graph information into paper encoding.
In the decoding process, we propose a two-stage decoder by first generating knowledge graph information of summary in the form of descriptive sentences, followed by generating the final summary.
arXiv Detail & Related papers (2022-09-09T14:20:59Z) - Hierarchical Sketch Induction for Paraphrase Generation [79.87892048285819]
We introduce Hierarchical Refinement Quantized Variational Autoencoders (HRQ-VAE), a method for learning decompositions of dense encodings.
We use HRQ-VAE to encode the syntactic form of an input sentence as a path through the hierarchy, allowing us to more easily predict syntactic sketches at test time.
arXiv Detail & Related papers (2022-03-07T15:28:36Z) - Hierarchical Bi-Directional Self-Attention Networks for Paper Review
Rating Recommendation [81.55533657694016]
We propose a Hierarchical bi-directional self-attention Network framework (HabNet) for paper review rating prediction and recommendation.
Specifically, we leverage the hierarchical structure of the paper reviews with three levels of encoders: sentence encoder (level one), intra-review encoder (level two) and inter-review encoder (level three)
We are able to identify useful predictors to make the final acceptance decision, as well as to help discover the inconsistency between numerical review ratings and text sentiment conveyed by reviewers.
arXiv Detail & Related papers (2020-11-02T08:07:50Z) - Keyphrase Extraction with Dynamic Graph Convolutional Networks and
Diversified Inference [50.768682650658384]
Keyphrase extraction (KE) aims to summarize a set of phrases that accurately express a concept or a topic covered in a given document.
Recent Sequence-to-Sequence (Seq2Seq) based generative framework is widely used in KE task, and it has obtained competitive performance on various benchmarks.
In this paper, we propose to adopt the Dynamic Graph Convolutional Networks (DGCN) to solve the above two problems simultaneously.
arXiv Detail & Related papers (2020-10-24T08:11:23Z) - Topic-Guided Abstractive Text Summarization: a Joint Learning Approach [19.623946402970933]
We introduce a new approach for abstractive text summarization, Topic-Guided Abstractive Summarization.
The idea is to incorporate neural topic modeling with a Transformer-based sequence-to-sequence (seq2seq) model in a joint learning framework.
arXiv Detail & Related papers (2020-10-20T14:45:25Z) - Cross-Thought for Sentence Encoder Pre-training [89.32270059777025]
Cross-Thought is a novel approach to pre-training sequence encoder.
We train a Transformer-based sequence encoder over a large set of short sequences.
Experiments on question answering and textual entailment tasks demonstrate that our pre-trained encoder can outperform state-of-the-art encoders.
arXiv Detail & Related papers (2020-10-07T21:02:41Z) - Improved Code Summarization via a Graph Neural Network [96.03715569092523]
In general, source code summarization techniques use the source code as input and outputs a natural language description.
We present an approach that uses a graph-based neural architecture that better matches the default structure of the AST to generate these summaries.
arXiv Detail & Related papers (2020-04-06T17:36:42Z) - Selective Attention Encoders by Syntactic Graph Convolutional Networks
for Document Summarization [21.351111598564987]
We propose a graph to connect the parsing trees from the sentences in a document and utilize the stacked graph convolutional networks (GCNs) to learn the syntactic representation for a document.
The proposed GCNs based selective attention approach outperforms the baselines and achieves the state-of-the-art performance on the dataset.
arXiv Detail & Related papers (2020-03-18T01:30:02Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.