Growing 3D Artefacts and Functional Machines with Neural Cellular Automata

• URL: http://arxiv.org/abs/2103.08737v1
• Date: Mon, 15 Mar 2021 21:51:04 GMT
• Title: Growing 3D Artefacts and Functional Machines with Neural Cellular Automata
• Authors: Shyam Sudhakaran, Djordje Grbic, Siyan Li, Adam Katona, Elias Najarro, Claire Glanois, Sebastian Risi
• Abstract summary: We propose an extension of NCAs to 3D, utilizing 3D convolutions in the proposed neural network architecture. We show that NCAs are capable of growing complex entities such as castles, apartment blocks, and trees, some of which are composed of over 3,000 blocks.
• Score: 6.0034482500242765
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Neural Cellular Automata (NCAs) have been proven effective in simulating morphogenetic processes, the continuous construction of complex structures from very few starting cells. Recent developments in NCAs lie in the 2D domain, namely reconstructing target images from a single pixel or infinitely growing 2D textures. In this work, we propose an extension of NCAs to 3D, utilizing 3D convolutions in the proposed neural network architecture. Minecraft is selected as the environment for our automaton since it allows the generation of both static structures and moving machines. We show that despite their simplicity, NCAs are capable of growing complex entities such as castles, apartment blocks, and trees, some of which are composed of over 3,000 blocks. Additionally, when trained for regeneration, the system is able to regrow parts of simple functional machines, significantly expanding the capabilities of simulated morphogenetic systems.

Related papers

• Multi-Texture Synthesis through Signal Responsive Neural Cellular Automata [44.99833362998488]
  We train a single NCA for the evolution of multiple textures, based on individual examples. Our solution provides texture information in the state of each cell, in the form of an internally coded genomic signal, which enables the NCA to generate the expected texture.
  arXiv  Detail & Related papers  (2024-07-08T14:36:20Z)
• CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assets [43.315487682462845]
  CLAY is a 3D geometry and material generator designed to transform human imagination into intricate 3D digital structures. At its core is a large-scale generative model composed of a multi-resolution Variational Autoencoder (VAE) and a minimalistic latent Diffusion Transformer (DiT) We demonstrate using CLAY for a range of controllable 3D asset creations, from sketchy conceptual designs to production ready assets with intricate details.
  arXiv  Detail & Related papers  (2024-05-30T05:57:36Z)
• Progress and Prospects in 3D Generative AI: A Technical Overview including 3D human [51.58094069317723]
  This paper aims to provide a comprehensive overview and summary of the relevant papers published mostly during the latter half year of 2023. It will begin by discussing the AI generated object models in 3D, followed by the generated 3D human models, and finally, the generated 3D human motions, culminating in a conclusive summary and a vision for the future.
  arXiv  Detail & Related papers  (2024-01-05T03:41:38Z)
• Mesh Neural Cellular Automata [62.101063045659906]
  We propose Mesh Neural Cellular Automata (MeshNCA), a method that directly synthesizes dynamic textures on 3D meshes without requiring any UV maps. Only trained on an Icosphere mesh, MeshNCA shows remarkable test-time generalization and can synthesize textures on unseen meshes in real time.
  arXiv  Detail & Related papers  (2023-11-06T01:54:37Z)
• Exploring Multiple Neighborhood Neural Cellular Automata (MNNCA) for Enhanced Texture Learning [0.0]
  Cellular Automata (CA) have long been foundational in simulating dynamical systems. Recent innovations have brought Neural Cellular Automata (NCA) into the realm of deep learning. NCA allows NCAs to be trained via gradient descent, enabling them to evolve into specific shapes, generate textures, and mimic behaviors such as swarming. Our research explores enhancing the NCA framework by incorporating multiple neighborhoods and introducing structured noise for seed states.
  arXiv  Detail & Related papers  (2023-10-27T15:16:19Z)
• Learning Versatile 3D Shape Generation with Improved AR Models [91.87115744375052]
  Auto-regressive (AR) models have achieved impressive results in 2D image generation by modeling joint distributions in the grid space. We propose the Improved Auto-regressive Model (ImAM) for 3D shape generation, which applies discrete representation learning based on a latent vector instead of volumetric grids.
  arXiv  Detail & Related papers  (2023-03-26T12:03:18Z)
• GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images [72.15855070133425]
  We introduce GET3D, a Generative model that directly generates Explicit Textured 3D meshes with complex topology, rich geometric details, and high-fidelity textures. GET3D is able to generate high-quality 3D textured meshes, ranging from cars, chairs, animals, motorbikes and human characters to buildings.
  arXiv  Detail & Related papers  (2022-09-22T17:16:19Z)
• Fast-GANFIT: Generative Adversarial Network for High Fidelity 3D Face Reconstruction [76.1612334630256]
  We harness the power of Generative Adversarial Networks (GANs) and Deep Convolutional Neural Networks (DCNNs) to reconstruct the facial texture and shape from single images. We demonstrate excellent results in photorealistic and identity preserving 3D face reconstructions and achieve for the first time, facial texture reconstruction with high-frequency details.
  arXiv  Detail & Related papers  (2021-05-16T16:35:44Z)
• Attention-based 3D Object Reconstruction from a Single Image [0.2519906683279153]
  We propose to substantially improve Occupancy Networks, a state-of-the-art method for 3D object reconstruction. We apply the concept of self-attention within the network's encoder in order to leverage complementary input features. We were able to improve the original work in 5.05% of mesh IoU, 0.83% of Normal Consistency, and more than 10X the Chamfer-L1 distance.
  arXiv  Detail & Related papers  (2020-08-11T14:51:18Z)
• Neural Cellular Automata Manifold [84.08170531451006]
  We show that the neural network architecture of the Neural Cellular Automata can be encapsulated in a larger NN. This allows us to propose a new model that encodes a manifold of NCA, each of them capable of generating a distinct image. In biological terms, our approach would play the role of the transcription factors, modulating the mapping of genes into specific proteins that drive cellular differentiation.
  arXiv  Detail & Related papers  (2020-06-22T11:41:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.