Abstract: The paper proposes a solution based on Generative Adversarial Network (GAN)
for solving jigsaw puzzles. The problem assumes that an image is cut into equal
square pieces, and asks to recover the image according to pieces information.
Conventional jigsaw solvers often determine piece relationships based on the
piece boundaries, which ignore the important semantic information. In this
paper, we propose JigsawGAN, a GAN-based self-supervised method for solving
jigsaw puzzles with unpaired images (with no prior knowledge of the initial
images). We design a multi-task pipeline that includes, (1) a classification
branch to classify jigsaw permutations, and (2) a GAN branch to recover
features to images with correct orders. The classification branch is
constrained by the pseudo-labels generated according to the shuffled pieces.
The GAN branch concentrates on the image semantic information, among which the
generator produces the natural images to fool the discriminator with
reassembled pieces, while the discriminator distinguishes whether a given image
belongs to the synthesized or the real target manifold. These two branches are
connected by a flow-based warp that is applied to warp features to correct
order according to the classification results. The proposed method can solve
jigsaw puzzles more efficiently by utilizing both semantic information and edge
information simultaneously. Qualitative and quantitative comparisons against
several leading prior methods demonstrate the superiority of our method.