Conditional Self-Attention for Query-based Summarization
- URL: http://arxiv.org/abs/2002.07338v1
- Date: Tue, 18 Feb 2020 02:22:31 GMT
- Title: Conditional Self-Attention for Query-based Summarization
- Authors: Yujia Xie, Tianyi Zhou, Yi Mao, Weizhu Chen
- Abstract summary: We propose textitconditional self-attention (CSA), a neural network module designed for conditional dependency modeling.
Experiments on Debatepedia and HotpotQA benchmark datasets show CSA consistently outperforms vanilla Transformer.
- Score: 49.616774159367516
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Self-attention mechanisms have achieved great success on a variety of NLP
tasks due to its flexibility of capturing dependency between arbitrary
positions in a sequence. For problems such as query-based summarization (Qsumm)
and knowledge graph reasoning where each input sequence is associated with an
extra query, explicitly modeling such conditional contextual dependencies can
lead to a more accurate solution, which however cannot be captured by existing
self-attention mechanisms. In this paper, we propose \textit{conditional
self-attention} (CSA), a neural network module designed for conditional
dependency modeling. CSA works by adjusting the pairwise attention between
input tokens in a self-attention module with the matching score of the inputs
to the given query. Thereby, the contextual dependencies modeled by CSA will be
highly relevant to the query. We further studied variants of CSA defined by
different types of attention. Experiments on Debatepedia and HotpotQA benchmark
datasets show CSA consistently outperforms vanilla Transformer and previous
models for the Qsumm problem.
Related papers
- An Empirical Comparison of LM-based Question and Answer Generation
Methods [79.31199020420827]
Question and answer generation (QAG) consists of generating a set of question-answer pairs given a context.
In this paper, we establish baselines with three different QAG methodologies that leverage sequence-to-sequence language model (LM) fine-tuning.
Experiments show that an end-to-end QAG model, which is computationally light at both training and inference times, is generally robust and outperforms other more convoluted approaches.
arXiv Detail & Related papers (2023-05-26T14:59:53Z) - Modeling Global Semantics for Question Answering over Knowledge Bases [16.341353183347664]
We present a relational graph convolutional network (RGCN)-based model gRGCN for semantic parsing in KBQA.
Experiments evaluated on benchmarks show that our model outperforms off-the-shelf models.
arXiv Detail & Related papers (2021-01-05T13:51:14Z) - A Hierarchical Reasoning Graph Neural Network for The Automatic Scoring
of Answer Transcriptions in Video Job Interviews [14.091472037847499]
We propose a Hierarchical Reasoning Graph Neural Network (HRGNN) for the automatic assessment of question-answer pairs.
We employ a semantic-level reasoning graph attention network to model the interaction states of the current QA session.
Finally, we propose a gated recurrent unit encoder to represent the temporal question-answer pairs for the final prediction.
arXiv Detail & Related papers (2020-12-22T12:27:45Z) - Learning from Lexical Perturbations for Consistent Visual Question
Answering [78.21912474223926]
Existing Visual Question Answering (VQA) models are often fragile and sensitive to input variations.
We propose a novel approach to address this issue based on modular networks, which creates two questions related by linguistic perturbations.
We also present VQA Perturbed Pairings (VQA P2), a new, low-cost benchmark and augmentation pipeline to create controllable linguistic variations.
arXiv Detail & Related papers (2020-11-26T17:38:03Z) - Generating Diverse and Consistent QA pairs from Contexts with
Information-Maximizing Hierarchical Conditional VAEs [62.71505254770827]
We propose a conditional variational autoencoder (HCVAE) for generating QA pairs given unstructured texts as contexts.
Our model obtains impressive performance gains over all baselines on both tasks, using only a fraction of data for training.
arXiv Detail & Related papers (2020-05-28T08:26:06Z) - Robust Question Answering Through Sub-part Alignment [53.94003466761305]
We model question answering as an alignment problem.
We train our model on SQuAD v1.1 and test it on several adversarial and out-of-domain datasets.
arXiv Detail & Related papers (2020-04-30T09:10:57Z) - Query Focused Multi-Document Summarization with Distant Supervision [88.39032981994535]
Existing work relies heavily on retrieval-style methods for estimating the relevance between queries and text segments.
We propose a coarse-to-fine modeling framework which introduces separate modules for estimating whether segments are relevant to the query.
We demonstrate that our framework outperforms strong comparison systems on standard QFS benchmarks.
arXiv Detail & Related papers (2020-04-06T22:35:19Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.