Multi-Texture Synthesis through Signal Responsive Neural Cellular Automata

• URL: http://arxiv.org/abs/2407.05991v2
• Date: Fri, 19 Jul 2024 08:17:44 GMT
• Title: Multi-Texture Synthesis through Signal Responsive Neural Cellular Automata
• Authors: Mirela-Magdalena Catrina, Ioana Cristina Plajer, Alexandra Baicoianu,
• Abstract summary: We train a single NCA for the evolution of multiple textures, based on individual examples. Our solution provides texture information in the state of each cell, in the form of an internally coded genomic signal, which enables the NCA to generate the expected texture.
• Score: 44.99833362998488
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Neural Cellular Automata (NCA) have proven to be effective in a variety of fields, with numerous biologically inspired applications. One of the fields, in which NCAs perform well is the generation of textures, modelling global patterns from local interactions governed by uniform and coherent rules. This paper aims to enhance the usability of NCAs in texture synthesis by addressing a shortcoming of current NCA architectures for texture generation, which requires separately trained NCA for each individual texture. In this work, we train a single NCA for the evolution of multiple textures, based on individual examples. Our solution provides texture information in the state of each cell, in the form of an internally coded genomic signal, which enables the NCA to generate the expected texture. Such a neural cellular automaton not only maintains its regenerative capability but also allows for interpolation between learned textures and supports grafting techniques. This demonstrates the ability to edit generated textures and the potential for them to merge and coexist within the same automaton. We also address questions related to the influence of the genomic information and the cost function on the evolution of the NCA.

Related papers

• Neural Texture Puppeteer: A Framework for Neural Geometry and Texture Rendering of Articulated Shapes, Enabling Re-Identification at Interactive Speed [2.8544822698499255]
  We present a neural rendering pipeline for textured articulated shapes that we call Neural Texture Puppeteer. A texture auto-encoder makes use of this information to encode textured images into a global latent code. Our method can be applied to endangered species where data is limited.
  arXiv  Detail & Related papers  (2023-11-28T10:51:05Z)
• Mesh Neural Cellular Automata [62.101063045659906]
  We propose Mesh Neural Cellular Automata (MeshNCA), a method that directly synthesizes dynamic textures on 3D meshes without requiring any UV maps. Only trained on an Icosphere mesh, MeshNCA shows remarkable test-time generalization and can synthesize textures on unseen meshes in real time.
  arXiv  Detail & Related papers  (2023-11-06T01:54:37Z)
• Exploring Multiple Neighborhood Neural Cellular Automata (MNNCA) for Enhanced Texture Learning [0.0]
  Cellular Automata (CA) have long been foundational in simulating dynamical systems. Recent innovations have brought Neural Cellular Automata (NCA) into the realm of deep learning. NCA allows NCAs to be trained via gradient descent, enabling them to evolve into specific shapes, generate textures, and mimic behaviors such as swarming. Our research explores enhancing the NCA framework by incorporating multiple neighborhoods and introducing structured noise for seed states.
  arXiv  Detail & Related papers  (2023-10-27T15:16:19Z)
• Mitigating Generation Shifts for Generalized Zero-Shot Learning [52.98182124310114]
  Generalized Zero-Shot Learning (GZSL) is the task of leveraging semantic information (e.g., attributes) to recognize the seen and unseen samples, where unseen classes are not observable during training. We propose a novel Generation Shifts Mitigating Flow framework for learning unseen data synthesis efficiently and effectively. Experimental results demonstrate that GSMFlow achieves state-of-the-art recognition performance in both conventional and generalized zero-shot settings.
  arXiv  Detail & Related papers  (2021-07-07T11:43:59Z)
• Texture Generation with Neural Cellular Automata [64.70093734012121]
  We learn a texture generator from a single template image. We make claims that the behaviour exhibited by the NCA model is a learned, distributed, local algorithm to generate a texture.
  arXiv  Detail & Related papers  (2021-05-15T22:05:46Z)
• Neural Cellular Automata Manifold [84.08170531451006]
  We show that the neural network architecture of the Neural Cellular Automata can be encapsulated in a larger NN. This allows us to propose a new model that encodes a manifold of NCA, each of them capable of generating a distinct image. In biological terms, our approach would play the role of the transcription factors, modulating the mapping of genes into specific proteins that drive cellular differentiation.
  arXiv  Detail & Related papers  (2020-06-22T11:41:57Z)
• Towards Analysis-friendly Face Representation with Scalable Feature and Texture Compression [113.30411004622508]
  We show that a universal and collaborative visual information representation can be achieved in a hierarchical way. Based on the strong generative capability of deep neural networks, the gap between the base feature layer and enhancement layer is further filled with the feature level texture reconstruction. To improve the efficiency of the proposed framework, the base layer neural network is trained in a multi-task manner.
  arXiv  Detail & Related papers  (2020-04-21T14:32:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.