Deepzzle: Solving Visual Jigsaw Puzzles with Deep Learning andShortest Path Optimization

• URL: http://arxiv.org/abs/2005.12548v1
• Date: Tue, 26 May 2020 07:19:54 GMT
• Title: Deepzzle: Solving Visual Jigsaw Puzzles with Deep Learning andShortest Path Optimization
• Authors: Marie-Morgane Paumard, David Picard, Hedi Tabia
• Abstract summary: We tackle the image reassembly problem with wide space between the fragments. We crop-square the fragments borders to compel our algorithm to learn from the content of the fragments. We notably investigate the effect of branch-cut in the graph of reassemblies.
• Score: 30.43614740245788
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We tackle the image reassembly problem with wide space between the fragments, in such a way that the patterns and colors continuity is mostly unusable. The spacing emulates the erosion of which the archaeological fragments suffer. We crop-square the fragments borders to compel our algorithm to learn from the content of the fragments. We also complicate the image reassembly by removing fragments and adding pieces from other sources. We use a two-step method to obtain the reassemblies: 1) a neural network predicts the positions of the fragments despite the gaps between them; 2) a graph that leads to the best reassemblies is made from these predictions. In this paper, we notably investigate the effect of branch-cut in the graph of reassemblies. We also provide a comparison with the literature, solve complex images reassemblies, explore at length the dataset, and propose a new metric that suits its specificities. Keywords: image reassembly, jigsaw puzzle, deep learning, graph, branch-cut, cultural heritage

Related papers

• Re-assembling the past: The RePAIR dataset and benchmark for real world 2D and 3D puzzle solving [46.073519734119266]
  Our dataset has unique properties that are uncommon to current benchmarks for 2D and 3D puzzle solving. The fragments and fractures are realistic, caused by a collapse of a fresco during a World War II bombing at the Pompeii archaeological park. The dataset is multi-modal providing high resolution images with characteristic pictorial elements.
  arXiv  Detail & Related papers  (2024-10-31T15:10:38Z)
• MULAN: A Multi Layer Annotated Dataset for Controllable Text-to-Image Generation [54.64194935409982]
  We introduce MuLAn: a novel dataset comprising over 44K MUlti-Layer-wise RGBA decompositions. MuLAn is the first photorealistic resource providing instance decomposition and spatial information for high quality images. We aim to encourage the development of novel generation and editing technology, in particular layer-wise solutions.
  arXiv  Detail & Related papers  (2024-04-03T14:58:00Z)
• Block and Detail: Scaffolding Sketch-to-Image Generation [65.56590359051634]
  We introduce a novel sketch-to-image tool that aligns with the iterative refinement process of artists. Our tool lets users sketch blocking strokes to coarsely represent the placement and form of objects and detail strokes to refine their shape and silhouettes. We develop a two-pass algorithm for generating high-fidelity images from such sketches at any point in the iterative process.
  arXiv  Detail & Related papers  (2024-02-28T07:09:31Z)
• GANzzle: Reframing jigsaw puzzle solving as a retrieval task using a generative mental image [15.132848477903314]
  We infer a mental image from all pieces, which a given piece can then be matched against avoiding the explosion. We learn how to reconstruct the image given a set of unordered pieces, allowing the model to learn a joint embedding space to match an encoding of each piece to the cropped layer of the generator. In doing so our model is puzzle size agnostic, in contrast to prior deep learning methods which are single size.
  arXiv  Detail & Related papers  (2022-07-12T16:02:00Z)
• Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localization [98.46318529630109]
  We take inspiration from traditional spectral segmentation methods by reframing image decomposition as a graph partitioning problem. We find that these eigenvectors already decompose an image into meaningful segments, and can be readily used to localize objects in a scene. By clustering the features associated with these segments across a dataset, we can obtain well-delineated, nameable regions.
  arXiv  Detail & Related papers  (2022-05-16T17:47:44Z)
• Relaxation Labeling Meets GANs: Solving Jigsaw Puzzles with Missing Borders [13.98838872235379]
  We propose JiGAN, a GAN-based method for solving Jigsaw puzzles with eroded or missing borders. We test the method on a large dataset of small puzzles and on three commonly used benchmark datasets to demonstrate the feasibility of the proposed approach.
  arXiv  Detail & Related papers  (2022-03-28T00:38:17Z)
• Unsupervised Part Discovery from Contrastive Reconstruction [90.88501867321573]
  The goal of self-supervised visual representation learning is to learn strong, transferable image representations. We propose an unsupervised approach to object part discovery and segmentation. Our method yields semantic parts consistent across fine-grained but visually distinct categories.
  arXiv  Detail & Related papers  (2021-11-11T17:59:42Z)
• Non-Rigid Puzzles [50.213265511586535]
  We present a non-rigid multi-part shape matching algorithm. We assume to be given a reference shape and its multiple parts undergoing a non-rigid deformation. Experimental results on synthetic as well as real scans demonstrate the effectiveness of our method.
  arXiv  Detail & Related papers  (2020-11-26T00:32:30Z)
• Using Graph Neural Networks to Reconstruct Ancient Documents [2.4366811507669124]
  We present a solution based on a Graph Neural Network, using pairwise patch information to assign labels to edges. This network classifies the relationship between a source and a target patch as being one of Up, Down, Left, Right or None. We show that our model is not only able to provide correct classifications at the edge-level, but also to generate partial or full reconstruction graphs from a set of patches.
  arXiv  Detail & Related papers  (2020-11-13T18:36:36Z)
• PuzzleNet: Scene Text Detection by Segment Context Graph Learning [9.701699882807251]
  We propose a novel decomposition-based method, termed Puzzle Networks (PuzzleNet), to address the challenging scene text detection task. By building segments as context graphs, MSGCN effectively employs segment context to predict combinations of segments. Our method can achieve better or comparable performance than current state-of-the-arts, which is beneficial from the exploitation of segment context graph.
  arXiv  Detail & Related papers  (2020-02-26T09:21:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.