JigsawGAN: Self-supervised Learning for Solving Jigsaw Puzzles with Generative Adversarial Networks

• URL: http://arxiv.org/abs/2101.07555v1
• Date: Tue, 19 Jan 2021 10:40:38 GMT
• Title: JigsawGAN: Self-supervised Learning for Solving Jigsaw Puzzles with Generative Adversarial Networks
• Authors: Ru Li, Shuaicheng Liu, Guangfu Wang, Guanghui Liu and Bing Zeng
• Abstract summary: The paper proposes a solution based on Generative Adversarial Network (GAN) for solving jigsaw puzzles. The proposed method can solve jigsaw puzzles more efficiently by utilizing both semantic information and edge information simultaneously.
• Score: 31.190344964881625
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The paper proposes a solution based on Generative Adversarial Network (GAN) for solving jigsaw puzzles. The problem assumes that an image is cut into equal square pieces, and asks to recover the image according to pieces information. Conventional jigsaw solvers often determine piece relationships based on the piece boundaries, which ignore the important semantic information. In this paper, we propose JigsawGAN, a GAN-based self-supervised method for solving jigsaw puzzles with unpaired images (with no prior knowledge of the initial images). We design a multi-task pipeline that includes, (1) a classification branch to classify jigsaw permutations, and (2) a GAN branch to recover features to images with correct orders. The classification branch is constrained by the pseudo-labels generated according to the shuffled pieces. The GAN branch concentrates on the image semantic information, among which the generator produces the natural images to fool the discriminator with reassembled pieces, while the discriminator distinguishes whether a given image belongs to the synthesized or the real target manifold. These two branches are connected by a flow-based warp that is applied to warp features to correct order according to the classification results. The proposed method can solve jigsaw puzzles more efficiently by utilizing both semantic information and edge information simultaneously. Qualitative and quantitative comparisons against several leading prior methods demonstrate the superiority of our method.

Related papers

• Accelerated Sub-Image Search For Variable-Size Patches Identification Based On Virtual Time Series Transformation And Segmentation [0.0]
  This paper addresses two tasks: (i) fixed-size objects such as hay bales to be identified in an aerial image for a given reference image of the object, and (ii) variable-size patches such as areas on fields requiring spot spraying or other handling are to be identified in an image for a given small-scale reference image. The exact number of similar sub-images is not known a priori.
  arXiv  Detail & Related papers  (2024-10-20T15:43:50Z)
• Patch-Based Deep Unsupervised Image Segmentation using Graph Cuts [0.0]
  We propose a patch-based unsupervised image segmentation strategy that bridges advances in unsupervised feature extraction with the algorithmic help of classical graph-based methods. We show that a simple convolutional neural network, trained to classify image patches, naturally leads to a state-of-the-art fully-convolutional unsupervised pixel-level segmenter.
  arXiv  Detail & Related papers  (2023-11-01T19:59:25Z)
• Learning to Annotate Part Segmentation with Gradient Matching [58.100715754135685]
  This paper focuses on tackling semi-supervised part segmentation tasks by generating high-quality images with a pre-trained GAN. In particular, we formulate the annotator learning as a learning-to-learn problem. We show that our method can learn annotators from a broad range of labelled images including real images, generated images, and even analytically rendered images.
  arXiv  Detail & Related papers  (2022-11-06T01:29:22Z)
• GANzzle: Reframing jigsaw puzzle solving as a retrieval task using a generative mental image [15.132848477903314]
  We infer a mental image from all pieces, which a given piece can then be matched against avoiding the explosion. We learn how to reconstruct the image given a set of unordered pieces, allowing the model to learn a joint embedding space to match an encoding of each piece to the cropped layer of the generator. In doing so our model is puzzle size agnostic, in contrast to prior deep learning methods which are single size.
  arXiv  Detail & Related papers  (2022-07-12T16:02:00Z)
• Ensembling with Deep Generative Views [72.70801582346344]
  generative models can synthesize "views" of artificial images that mimic real-world variations, such as changes in color or pose. Here, we investigate whether such views can be applied to real images to benefit downstream analysis tasks such as image classification. We use StyleGAN2 as the source of generative augmentations and investigate this setup on classification tasks involving facial attributes, cat faces, and cars.
  arXiv  Detail & Related papers  (2021-04-29T17:58:35Z)
• White Box Methods for Explanations of Convolutional Neural Networks in Image Classification Tasks [3.3959642559854357]
  Convolutional Neural Networks (CNNs) have demonstrated state of the art performance for the task of image classification. Several approaches have been proposed to explain to understand the reasoning behind a prediction made by a network. We focus primarily on white box methods that leverage the information of the internal architecture of a network to explain its decision.
  arXiv  Detail & Related papers  (2021-04-06T14:40:00Z)
• Convolutional Neural Networks from Image Markers [62.997667081978825]
  Feature Learning from Image Markers (FLIM) was recently proposed to estimate convolutional filters, with no backpropagation, from strokes drawn by a user on very few images. This paper extends FLIM for fully connected layers and demonstrates it on different image classification problems. The results show that FLIM-based convolutional neural networks can outperform the same architecture trained from scratch by backpropagation.
  arXiv  Detail & Related papers  (2020-12-15T22:58:23Z)
• Weakly-Supervised Semantic Segmentation by Iterative Affinity Learning [86.45526827323954]
  Weakly-supervised semantic segmentation is a challenging task as no pixel-wise label information is provided for training. We propose an iterative algorithm to learn such pairwise relations. We show that the proposed algorithm performs favorably against the state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-02-19T10:32:03Z)
• Multi-Channel Attention Selection GANs for Guided Image-to-Image Translation [148.9985519929653]
  We propose a novel model named Multi-Channel Attention Selection Generative Adversarial Network (SelectionGAN) for guided image-to-image translation. The proposed framework and modules are unified solutions and can be applied to solve other generation tasks such as semantic image synthesis.
  arXiv  Detail & Related papers  (2020-02-03T23:17:10Z)
• Image Embedded Segmentation: Uniting Supervised and Unsupervised Objectives for Segmenting Histopathological Images [0.0]
  This paper presents a new regularization method to train a fully convolutional network for semantic tissue segmentation. It relies on the benefit of unsupervised learning, in the form of image reconstruction, for network training. Our experiments demonstrate that it leads to better segmentation results in these datasets, compared to its counterparts.
  arXiv  Detail & Related papers  (2020-01-30T08:09:38Z)
• OneGAN: Simultaneous Unsupervised Learning of Conditional Image Generation, Foreground Segmentation, and Fine-Grained Clustering [100.32273175423146]
  We present a method for simultaneously learning, in an unsupervised manner, a conditional image generator, foreground extraction and segmentation, and object removal and background completion. The method combines a Geneversarative Adrial Network and a Variational Auto-Encoder, with multiple encoders, generators and discriminators, and benefits from solving all tasks at once.
  arXiv  Detail & Related papers  (2019-12-31T18:15:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.