Abstract: The task of rationalization aims to extract pieces of input text as
rationales to justify neural network predictions on text classification tasks.
By definition, rationales represent key text pieces used for prediction and
thus should have similar classification feature distribution compared to the
original input text. However, previous methods mainly focused on maximizing the
mutual information between rationales and labels while neglecting the
relationship between rationales and input text. To address this issue, we
propose a novel rationalization method that matches the distributions of
rationales and input text in both the feature space and output space.
Empirically, the proposed distribution matching approach consistently
outperforms previous methods by a large margin. Our data and code are