Abstract: A sequence-to-sequence learning with neural networks has empirically proven
to be an effective framework for Chinese Spelling Correction (CSC), which takes
a sentence with some spelling errors as input and outputs the corrected one.
However, CSC models may fail to correct spelling errors covered by the
confusion sets, and also will encounter unseen ones. We propose a method, which
continually identifies the weak spots of a model to generate more valuable
training instances, and apply a task-specific pre-training strategy to enhance
the model. The generated adversarial examples are gradually added to the
training set. Experimental results show that such an adversarial training
method combined with the pretraining strategy can improve both the
generalization and robustness of multiple CSC models across three different
datasets, achieving stateof-the-art performance for CSC task.