Deepzzle: Solving Visual Jigsaw Puzzles with Deep Learning andShortest
Path Optimization
- URL: http://arxiv.org/abs/2005.12548v1
- Date: Tue, 26 May 2020 07:19:54 GMT
- Title: Deepzzle: Solving Visual Jigsaw Puzzles with Deep Learning andShortest
Path Optimization
- Authors: Marie-Morgane Paumard, David Picard, Hedi Tabia
- Abstract summary: We tackle the image reassembly problem with wide space between the fragments.
We crop-square the fragments borders to compel our algorithm to learn from the content of the fragments.
We notably investigate the effect of branch-cut in the graph of reassemblies.
- Score: 30.43614740245788
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We tackle the image reassembly problem with wide space between the fragments,
in such a way that the patterns and colors continuity is mostly unusable. The
spacing emulates the erosion of which the archaeological fragments suffer. We
crop-square the fragments borders to compel our algorithm to learn from the
content of the fragments. We also complicate the image reassembly by removing
fragments and adding pieces from other sources. We use a two-step method to
obtain the reassemblies: 1) a neural network predicts the positions of the
fragments despite the gaps between them; 2) a graph that leads to the best
reassemblies is made from these predictions. In this paper, we notably
investigate the effect of branch-cut in the graph of reassemblies. We also
provide a comparison with the literature, solve complex images reassemblies,
explore at length the dataset, and propose a new metric that suits its
specificities.
Keywords: image reassembly, jigsaw puzzle, deep learning, graph, branch-cut,
cultural heritage
Related papers
- MULAN: A Multi Layer Annotated Dataset for Controllable Text-to-Image Generation [54.64194935409982]
We introduce MuLAn: a novel dataset comprising over 44K MUlti-Layer-wise RGBA decompositions.
MuLAn is the first photorealistic resource providing instance decomposition and spatial information for high quality images.
We aim to encourage the development of novel generation and editing technology, in particular layer-wise solutions.
arXiv Detail & Related papers (2024-04-03T14:58:00Z) - Batch-based Model Registration for Fast 3D Sherd Reconstruction [74.55975819488404]
3D reconstruction techniques have widely been used for digital documentation of archaeological fragments.
We aim to develop a portable, high- throughput, and accurate reconstruction system for efficient digitization of fragments excavated in archaeological sites.
We develop a new batch-based matching algorithm that pairs the front and back sides of the fragments, and a new Bilateral Boundary ICP algorithm that can register partial scans sharing very narrow overlapping regions.
arXiv Detail & Related papers (2022-11-13T13:08:59Z) - GANzzle: Reframing jigsaw puzzle solving as a retrieval task using a
generative mental image [15.132848477903314]
We infer a mental image from all pieces, which a given piece can then be matched against avoiding the explosion.
We learn how to reconstruct the image given a set of unordered pieces, allowing the model to learn a joint embedding space to match an encoding of each piece to the cropped layer of the generator.
In doing so our model is puzzle size agnostic, in contrast to prior deep learning methods which are single size.
arXiv Detail & Related papers (2022-07-12T16:02:00Z) - Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised
Semantic Segmentation and Localization [98.46318529630109]
We take inspiration from traditional spectral segmentation methods by reframing image decomposition as a graph partitioning problem.
We find that these eigenvectors already decompose an image into meaningful segments, and can be readily used to localize objects in a scene.
By clustering the features associated with these segments across a dataset, we can obtain well-delineated, nameable regions.
arXiv Detail & Related papers (2022-05-16T17:47:44Z) - Relaxation Labeling Meets GANs: Solving Jigsaw Puzzles with Missing
Borders [13.98838872235379]
We propose JiGAN, a GAN-based method for solving Jigsaw puzzles with eroded or missing borders.
We test the method on a large dataset of small puzzles and on three commonly used benchmark datasets to demonstrate the feasibility of the proposed approach.
arXiv Detail & Related papers (2022-03-28T00:38:17Z) - Unsupervised Part Discovery from Contrastive Reconstruction [90.88501867321573]
The goal of self-supervised visual representation learning is to learn strong, transferable image representations.
We propose an unsupervised approach to object part discovery and segmentation.
Our method yields semantic parts consistent across fine-grained but visually distinct categories.
arXiv Detail & Related papers (2021-11-11T17:59:42Z) - Learning Co-segmentation by Segment Swapping for Retrieval and Discovery [67.6609943904996]
The goal of this work is to efficiently identify visually similar patterns from a pair of images.
We generate synthetic training pairs by selecting object segments in an image and copy-pasting them into another image.
We show our approach provides clear improvements for artwork details retrieval on the Brueghel dataset.
arXiv Detail & Related papers (2021-10-29T16:51:16Z) - Non-Rigid Puzzles [50.213265511586535]
We present a non-rigid multi-part shape matching algorithm.
We assume to be given a reference shape and its multiple parts undergoing a non-rigid deformation.
Experimental results on synthetic as well as real scans demonstrate the effectiveness of our method.
arXiv Detail & Related papers (2020-11-26T00:32:30Z) - Using Graph Neural Networks to Reconstruct Ancient Documents [2.4366811507669124]
We present a solution based on a Graph Neural Network, using pairwise patch information to assign labels to edges.
This network classifies the relationship between a source and a target patch as being one of Up, Down, Left, Right or None.
We show that our model is not only able to provide correct classifications at the edge-level, but also to generate partial or full reconstruction graphs from a set of patches.
arXiv Detail & Related papers (2020-11-13T18:36:36Z) - Pictorial and apictorial polygonal jigsaw puzzles: The lazy caterer
model, properties, and solvers [14.08706290287121]
We formalize a new type of jigsaw puzzle where the pieces are general convex polygons generated by cutting through a global polygonal shape/image with an arbitrary number of straight cuts.
We analyze the theoretical properties of such puzzles, including the inherent challenges in solving them once pieces are contaminated with geometrical noise.
arXiv Detail & Related papers (2020-08-17T22:07:40Z) - PuzzleNet: Scene Text Detection by Segment Context Graph Learning [9.701699882807251]
We propose a novel decomposition-based method, termed Puzzle Networks (PuzzleNet), to address the challenging scene text detection task.
By building segments as context graphs, MSGCN effectively employs segment context to predict combinations of segments.
Our method can achieve better or comparable performance than current state-of-the-arts, which is beneficial from the exploitation of segment context graph.
arXiv Detail & Related papers (2020-02-26T09:21:05Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.