Abstract: Recent studies show that Question Answering (QA) based on Answer Sentence
Selection (AS2) can be improved by generating an improved answer from the top-k
ranked answer sentences (termed GenQA). This allows for synthesizing the
information from multiple candidates into a concise, natural-sounding answer.
However, creating large-scale supervised training data for GenQA models is very
challenging. In this paper, we propose to train a GenQA model by transferring
knowledge from a trained AS2 model, to overcome the aforementioned issue.
First, we use an AS2 model to produce a ranking over answer candidates for a
set of questions. Then, we use the top ranked candidate as the generation
target, and the next k top ranked candidates as context for training a GenQA
model. We also propose to use the AS2 model prediction scores for loss
weighting and score-conditioned input/output shaping, to aid the knowledge
transfer. Our evaluation on three public and one large industrial datasets
demonstrates the superiority of our approach over the AS2 baseline, and GenQA
trained using supervised data.