Neural inverse procedural modeling of knitting yarns from images
- URL: http://arxiv.org/abs/2303.00154v1
- Date: Wed, 1 Mar 2023 00:56:39 GMT
- Title: Neural inverse procedural modeling of knitting yarns from images
- Authors: Elena Trunz, Jonathan Klein, Jan M\"uller, Lukas Bode, Ralf Sarlette,
Michael Weinmann, Reinhard Klein
- Abstract summary: We show that the complexity of yarn structures can be better encountered in terms of ensembles of networks that focus on individual characteristics.
We demonstrate that the combination of a carefully designed parametric, procedural yarn model with respective network ensembles as well as loss functions even allows robust parameter inference.
- Score: 6.114281140793954
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: We investigate the capabilities of neural inverse procedural modeling to
infer high-quality procedural yarn models with fiber-level details from single
images of depicted yarn samples. While directly inferring all parameters of the
underlying yarn model based on a single neural network may seem an intuitive
choice, we show that the complexity of yarn structures in terms of twisting and
migration characteristics of the involved fibers can be better encountered in
terms of ensembles of networks that focus on individual characteristics. We
analyze the effect of different loss functions including a parameter loss to
penalize the deviation of inferred parameters to ground truth annotations, a
reconstruction loss to enforce similar statistics of the image generated for
the estimated parameters in comparison to training images as well as an
additional regularization term to explicitly penalize deviations between latent
codes of synthetic images and the average latent code of real images in the
latent space of the encoder. We demonstrate that the combination of a carefully
designed parametric, procedural yarn model with respective network ensembles as
well as loss functions even allows robust parameter inference when solely
trained on synthetic data. Since our approach relies on the availability of a
yarn database with parameter annotations and we are not aware of such a
respectively available dataset, we additionally provide, to the best of our
knowledge, the first dataset of yarn images with annotations regarding the
respective yarn parameters. For this purpose, we use a novel yarn generator
that improves the realism of the produced results over previous approaches.
Related papers
- RANRAC: Robust Neural Scene Representations via Random Ray Consensus [12.161889666145127]
RANdom RAy Consensus (RANRAC) is an efficient approach to eliminate the effect of inconsistent data.
We formulate a fuzzy adaption of the RANSAC paradigm, enabling its application to large scale models.
Results indicate significant improvements compared to state-of-the-art robust methods for novel-view synthesis.
arXiv Detail & Related papers (2023-12-15T13:33:09Z) - Regularization, early-stopping and dreaming: a Hopfield-like setup to
address generalization and overfitting [0.0]
We look for optimal network parameters by applying a gradient descent over a regularized loss function.
Within this framework, the optimal neuron-interaction matrices correspond to Hebbian kernels revised by a reiterated unlearning protocol.
arXiv Detail & Related papers (2023-08-01T15:04:30Z) - Tunable Convolutions with Parametric Multi-Loss Optimization [5.658123802733283]
Behavior of neural networks is irremediably determined by the specific loss and data used during training.
It is often desirable to tune the model at inference time based on external factors such as preferences of the user or dynamic characteristics of the data.
This is especially important to balance the perception-distortion trade-off of ill-posed image-to-image translation tasks.
arXiv Detail & Related papers (2023-04-03T11:36:10Z) - DeepDC: Deep Distance Correlation as a Perceptual Image Quality
Evaluator [53.57431705309919]
ImageNet pre-trained deep neural networks (DNNs) show notable transferability for building effective image quality assessment (IQA) models.
We develop a novel full-reference IQA (FR-IQA) model based exclusively on pre-trained DNN features.
We conduct comprehensive experiments to demonstrate the superiority of the proposed quality model on five standard IQA datasets.
arXiv Detail & Related papers (2022-11-09T14:57:27Z) - Learning to Learn with Generative Models of Neural Network Checkpoints [71.06722933442956]
We construct a dataset of neural network checkpoints and train a generative model on the parameters.
We find that our approach successfully generates parameters for a wide range of loss prompts.
We apply our method to different neural network architectures and tasks in supervised and reinforcement learning.
arXiv Detail & Related papers (2022-09-26T17:59:58Z) - Self-Similarity Priors: Neural Collages as Differentiable Fractal
Representations [73.14227103400964]
We investigate the role of learning in the automated discovery of self-similarity and in its utilization for downstream tasks.
We design a novel class of implicit operators, Neural Collages, which represent data as the parameters of a self-referential, structured transformation.
We investigate how to leverage the representations produced by Neural Collages in various tasks, including data compression and generation.
arXiv Detail & Related papers (2022-04-15T22:54:23Z) - Conditional Variational Autoencoder for Learned Image Reconstruction [5.487951901731039]
We develop a novel framework that approximates the posterior distribution of the unknown image at each query observation.
It handles implicit noise models and priors, it incorporates the data formation process (i.e., the forward operator), and the learned reconstructive properties are transferable between different datasets.
arXiv Detail & Related papers (2021-10-22T10:02:48Z) - Scene Synthesis via Uncertainty-Driven Attribute Synchronization [52.31834816911887]
This paper introduces a novel neural scene synthesis approach that can capture diverse feature patterns of 3D scenes.
Our method combines the strength of both neural network-based and conventional scene synthesis approaches.
arXiv Detail & Related papers (2021-08-30T19:45:07Z) - Descriptive Modeling of Textiles using FE Simulations and Deep Learning [0.0]
We propose a novel and fully automated method for extracting the yarn geometrical features in woven composites.
The proposed approach employs two deep neural network architectures (U-Net and Mask RCNN)
Experimental results show that our method is accurate and robust for performing yarn instance segmentation on CT images.
arXiv Detail & Related papers (2021-06-26T09:32:24Z) - 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment
Feedback Loop [128.07841893637337]
Regression-based methods have recently shown promising results in reconstructing human meshes from monocular images.
Minor deviation in parameters may lead to noticeable misalignment between the estimated meshes and image evidences.
We propose a Pyramidal Mesh Alignment Feedback (PyMAF) loop to leverage a feature pyramid and rectify the predicted parameters.
arXiv Detail & Related papers (2021-03-30T17:07:49Z) - SIR: Self-supervised Image Rectification via Seeing the Same Scene from
Multiple Different Lenses [82.56853587380168]
We propose a novel self-supervised image rectification (SIR) method based on an important insight that the rectified results of distorted images of the same scene from different lens should be the same.
We leverage a differentiable warping module to generate the rectified images and re-distorted images from the distortion parameters.
Our method achieves comparable or even better performance than the supervised baseline method and representative state-of-the-art methods.
arXiv Detail & Related papers (2020-11-30T08:23:25Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.