Generative Escher Meshes

• URL: http://arxiv.org/abs/2309.14564v4
• Date: Mon, 17 Jun 2024 23:57:57 GMT
• Title: Generative Escher Meshes
• Authors: Noam Aigerman, Thibault Groueix,
• Abstract summary: This paper proposes a fully-automatic, text-guided generative method for producing perfectly-repeating, periodic, tile-able 2D imagery. In contrast to square texture images that are seamless when tiled, our method generates non-square tilings. We show our method is able to produce plausible, appealing results, with non-trivial tiles, for a variety of different periodic patterns.
• Score: 14.29301974658956
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper proposes a fully-automatic, text-guided generative method for producing perfectly-repeating, periodic, tile-able 2D imagery, such as the one seen on floors, mosaics, ceramics, and the work of M.C. Escher. In contrast to square texture images that are seamless when tiled, our method generates non-square tilings which comprise solely of repeating copies of the same object. It achieves this by optimizing both geometry and texture of a 2D mesh, yielding a non-square tile in the shape and appearance of the desired object, with close to no additional background details, that can tile the plane without gaps nor overlaps. We enable optimization of the tile's shape by an unconstrained, differentiable parameterization of the space of all valid tileable meshes for given boundary conditions stemming from a symmetry group. Namely, we construct a differentiable family of linear systems derived from a 2D mesh-mapping technique - Orbifold Tutte Embedding - by considering the mesh's Laplacian matrix as differentiable parameters. We prove that the solution space of these linear systems is exactly all possible valid tiling configurations, thereby providing an end-to-end differentiable representation for the entire space of valid tiles. We render the textured mesh via a differentiable renderer, and leverage a pre-trained image diffusion model to induce a loss on the resulting image, updating the mesh's parameters so as to make its appearance match the text prompt. We show our method is able to produce plausible, appealing results, with non-trivial tiles, for a variety of different periodic tiling patterns.

Related papers

• SpaceMesh: A Continuous Representation for Learning Manifold Surface Meshes [61.110517195874074]
  We present a scheme to directly generate manifold, polygonal meshes of complex connectivity as the output of a neural network. Our key innovation is to define a continuous latent connectivity space at each mesh, which implies the discrete mesh. In applications, this approach not only yields high-quality outputs from generative models, but also enables directly learning challenging geometry processing tasks such as mesh repair.
  arXiv  Detail & Related papers  (2024-09-30T17:59:03Z)
• Flexible Isosurface Extraction for Gradient-Based Mesh Optimization [65.76362454554754]
  This work considers gradient-based mesh optimization, where we iteratively optimize for a 3D surface mesh by representing it as the isosurface of a scalar field. We introduce FlexiCubes, an isosurface representation specifically designed for optimizing an unknown mesh with respect to geometric, visual, or even physical objectives.
  arXiv  Detail & Related papers  (2023-08-10T06:40:19Z)
• MeshDiffusion: Score-based Generative 3D Mesh Modeling [68.40770889259143]
  We consider the task of generating realistic 3D shapes for automatic scene generation and physical simulation. We take advantage of the graph structure of meshes and use a simple yet very effective generative modeling method to generate 3D meshes. Specifically, we represent meshes with deformable tetrahedral grids, and then train a diffusion model on this direct parametrization.
  arXiv  Detail & Related papers  (2023-03-14T17:59:01Z)
• NeuralMeshing: Differentiable Meshing of Implicit Neural Representations [63.18340058854517]
  We propose a novel differentiable meshing algorithm for extracting surface meshes from neural implicit representations. Our method produces meshes with regular tessellation patterns and fewer triangle faces compared to existing methods.
  arXiv  Detail & Related papers  (2022-10-05T16:52:25Z)
• EigenFairing: 3D Model Fairing using Image Coherence [0.884755712094096]
  A surface is often modeled as a triangulated mesh of 3D points and textures associated with faces of the mesh. When the points do not lie at critical points of maximum curvature or discontinuities of the real surface, faces of the mesh do not lie close to the modeled surface. This paper presents a technique for perfecting the 3D surface model by repositioning its vertices so that it is coherent with a set of observed images of the object.
  arXiv  Detail & Related papers  (2022-06-10T18:13:19Z)
• Deep Rectangling for Image Stitching: A Learning Baseline [57.76737888499145]
  We build the first image stitching rectangling dataset with a large diversity in irregular boundaries and scenes. Experiments demonstrate our superiority over traditional methods both quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2022-03-08T03:34:10Z)
• SeamlessGAN: Self-Supervised Synthesis of Tileable Texture Maps [3.504542161036043]
  We present SeamlessGAN, a method capable of automatically generating tileable texture maps from a single input exemplar. In contrast to most existing methods, focused solely on solving the synthesis problem, our work tackles both problems, synthesis and tileability, simultaneously.
  arXiv  Detail & Related papers  (2022-01-13T18:24:26Z)
• Differentiable Surface Triangulation [40.13834693745158]
  We present a differentiable surface triangulation that enables optimization for any per-vertex or per-face differentiable objective function over the space of underlying surface triangulations. Our method builds on the result that any 2D triangulation can be achieved by a suitably weighted Delaunay triangulation. We extend the algorithm to 3D by decomposing shapes into developable sets and differentiably meshing each set with suitable boundary constraints.
  arXiv  Detail & Related papers  (2021-09-22T12:42:43Z)
• DeepMesh: Differentiable Iso-Surface Extraction [53.77622255726208]
  We introduce a differentiable way to produce explicit surface mesh representations from Deep Implicit Fields. Our key insight is that by reasoning on how implicit field perturbations impact local surface geometry, one can ultimately differentiate the 3D location of surface samples. We exploit this to define DeepMesh -- end-to-end differentiable mesh representation that can vary its topology.
  arXiv  Detail & Related papers  (2021-06-20T20:12:41Z)
• Efficient texture mapping via a non-iterative global texture alignment [0.0]
  We present a non-iterative method for seamless texture reconstruction of a given 3D scene. Our method finds the best texture alignment in a single shot using a global optimisation framework. Experimental results demonstrate low computational complexity and outperformance compared to other alignment methods.
  arXiv  Detail & Related papers  (2020-11-02T10:24:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.