$\mu$NCA: Texture Generation with Ultra-Compact Neural Cellular Automata

• URL: http://arxiv.org/abs/2111.13545v1
• Date: Fri, 26 Nov 2021 15:26:37 GMT
• Title: $\mu$NCA: Texture Generation with Ultra-Compact Neural Cellular Automata
• Abstract summary: We study the problem of example-based procedural texture synthesis using highly compact models. We use differentiable programming to train a generative process, parameterised by a recurrent Neural Cellular Automata (NCA) rule.
• Score: 79.00163348781422
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We study the problem of example-based procedural texture synthesis using highly compact models. Given a sample image, we use differentiable programming to train a generative process, parameterised by a recurrent Neural Cellular Automata (NCA) rule. Contrary to the common belief that neural networks should be significantly over-parameterised, we demonstrate that our model architecture and training procedure allows for representing complex texture patterns using just a few hundred learned parameters, making their expressivity comparable to hand-engineered procedural texture generating programs. The smallest models from the proposed $\mu$NCA family scale down to 68 parameters. When using quantisation to one byte per parameter, proposed models can be shrunk to a size range between 588 and 68 bytes. Implementation of a texture generator that uses these parameters to produce images is possible with just a few lines of GLSL or C code.

Related papers

• SimpleProc: Fully Procedural Synthetic Data from Simple Rules for Multi-View Stereo [59.11389699606293]
  We show that we can generate effective training data using SimpleProc: a new, fully procedural generator driven by a very small set of rules.<n>At a modest scale of 8,000 images, our approach achieves superior results compared to manually curated images from games and real-world objects.<n>When scaled to 352,000 images, our method yields performance comparable to--and in several benchmarks, exceeding--models trained on over 692,000 manually curated images.
  arXiv  Detail & Related papers  (2026-04-06T17:59:05Z)
• Algorithmic Simplification of Neural Networks with Mosaic-of-Motifs [10.533982886956002]
  Methods like pruning, quantization, and knowledge distillation have been used to achieve massive reductions in the number of model parameters.<n>We take up the perspective of algorithmic complexity to explain this behavior.
  arXiv  Detail & Related papers  (2026-02-16T16:30:38Z)
• Masked Generative Nested Transformers with Decode Time Scaling [21.34984197218021]
  In this work, we aim to address the bottleneck of inference computational efficiency in visual generation algorithms. We design a decode time model scaling schedule to utilize compute effectively, and we can cache and reuse some of the computation. Our experiments show that with almost $3times$ less compute than baseline, our model obtains competitive performance.
  arXiv  Detail & Related papers  (2025-02-01T09:41:01Z)
• Computer Vision Model Compression Techniques for Embedded Systems: A Survey [75.38606213726906]
  This paper covers the main model compression techniques applied for computer vision tasks. We present the characteristics of compression subareas, compare different approaches, and discuss how to choose the best technique. We also share codes to assist researchers and new practitioners in overcoming initial implementation challenges.
  arXiv  Detail & Related papers  (2024-08-15T16:41:55Z)
• MCNC: Manifold-Constrained Reparameterization for Neural Compression [21.70510507535041]
  We present a novel model compression method, which we term Manifold-Constrained Neural Compression (MCNC) By constraining the parameter space to our proposed manifold, we can identify high-quality solutions. Our method significantly outperforms state-of-the-art baselines in terms of compression, accuracy, and/or model reconstruction time.
  arXiv  Detail & Related papers  (2024-06-27T16:17:26Z)
• MatFormer: Nested Transformer for Elastic Inference [91.45687988953435]
  MatFormer is a novel Transformer architecture designed to provide elastic inference across diverse deployment constraints. MatFormer achieves this by incorporating a nested Feed Forward Network (FFN) block structure within a standard Transformer model. We show that a 850M decoder-only MatFormer language model (MatLM) allows us to extract multiple smaller models spanning from 582M to 850M parameters.
  arXiv  Detail & Related papers  (2023-10-11T17:57:14Z)
• ClusTR: Exploring Efficient Self-attention via Clustering for Vision Transformers [70.76313507550684]
  We propose a content-based sparse attention method, as an alternative to dense self-attention. Specifically, we cluster and then aggregate key and value tokens, as a content-based method of reducing the total token count. The resulting clustered-token sequence retains the semantic diversity of the original signal, but can be processed at a lower computational cost.
  arXiv  Detail & Related papers  (2022-08-28T04:18:27Z)
• Bessel Equivariant Networks for Inversion of Transmission Effects in Multi-Mode Optical Fibres [3.2981146586835703]
  We develop a new type of model for solving the task of inverting the transmission effects of multi-mode optical fibres. We use the azimuthal correlations known to exist in fibre speckle patterns to account for the difference in spatial arrangement between input and speckle patterns. This model can scale to previously unachievable resolutions of imaging with multi-mode optical fibres and is demonstrated on $256 times 256$ pixel images.
  arXiv  Detail & Related papers  (2022-07-26T12:29:12Z)
• Variable Bitrate Neural Fields [75.24672452527795]
  We present a dictionary method for compressing feature grids, reducing their memory consumption by up to 100x. We formulate the dictionary optimization as a vector-quantized auto-decoder problem which lets us learn end-to-end discrete neural representations in a space where no direct supervision is available.
  arXiv  Detail & Related papers  (2022-06-15T17:58:34Z)
• Recurrent Parameter Generators [42.159272098922685]
  We present a generic method for recurrently using the same parameters for many different convolution layers to build a deep network. We demonstrate how to build a one-layer neural network to achieve similar performance compared to other traditional CNN models.
  arXiv  Detail & Related papers  (2021-07-15T04:23:59Z)
• Locally Masked Convolution for Autoregressive Models [107.4635841204146]
  LMConv is a simple modification to the standard 2D convolution that allows arbitrary masks to be applied to the weights at each location in the image. We learn an ensemble of distribution estimators that share parameters but differ in generation order, achieving improved performance on whole-image density estimation.
  arXiv  Detail & Related papers  (2020-06-22T17:59:07Z)
• Segmentation and Recovery of Superquadric Models using Convolutional Neural Networks [2.454342521577328]
  We present a (two-stage) approach built around convolutional neural networks (CNNs) In the first stage, our approach uses a Mask RCNN model to identify superquadric-like structures in depth scenes. We are able to describe complex structures with a small number of interpretable parameters.
  arXiv  Detail & Related papers  (2020-01-28T18:17:48Z)
• Model Fusion via Optimal Transport [64.13185244219353]
  We present a layer-wise model fusion algorithm for neural networks. We show that this can successfully yield "one-shot" knowledge transfer between neural networks trained on heterogeneous non-i.i.d. data.
  arXiv  Detail & Related papers  (2019-10-12T22:07:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.