Neural Cloth Simulation
- URL: http://arxiv.org/abs/2212.11220v1
- Date: Tue, 13 Dec 2022 16:05:59 GMT
- Title: Neural Cloth Simulation
- Authors: Hugo Bertiche, Meysam Madadi and Sergio Escalera
- Abstract summary: We present a framework for the garment animation problem through unsupervised deep learning inspired in physically based simulation.
We propose the first methodology able to learn realistic cloth dynamics unsupervisedly.
We show it also allows to control the level of motion in the predictions.
- Score: 41.42019559241777
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: We present a general framework for the garment animation problem through
unsupervised deep learning inspired in physically based simulation. Existing
trends in the literature already explore this possibility. Nonetheless, these
approaches do not handle cloth dynamics. Here, we propose the first methodology
able to learn realistic cloth dynamics unsupervisedly, and henceforth, a
general formulation for neural cloth simulation. The key to achieve this is to
adapt an existing optimization scheme for motion from simulation based
methodologies to deep learning. Then, analyzing the nature of the problem, we
devise an architecture able to automatically disentangle static and dynamic
cloth subspaces by design. We will show how this improves model performance.
Additionally, this opens the possibility of a novel motion augmentation
technique that greatly improves generalization. Finally, we show it also allows
to control the level of motion in the predictions. This is a useful, never seen
before, tool for artists. We provide of detailed analysis of the problem to
establish the bases of neural cloth simulation and guide future research into
the specifics of this domain.
Related papers
- Deep Learning Through A Telescoping Lens: A Simple Model Provides Empirical Insights On Grokking, Gradient Boosting & Beyond [61.18736646013446]
In pursuit of a deeper understanding of its surprising behaviors, we investigate the utility of a simple yet accurate model of a trained neural network.
Across three case studies, we illustrate how it can be applied to derive new empirical insights on a diverse range of prominent phenomena.
arXiv Detail & Related papers (2024-10-31T22:54:34Z) - Neural Material Adaptor for Visual Grounding of Intrinsic Dynamics [48.99021224773799]
We propose the Neural Material Adaptor (NeuMA), which integrates existing physical laws with learned corrections.
We also propose Particle-GS, a particle-driven 3D Gaussian Splatting variant that bridges simulation and observed images.
arXiv Detail & Related papers (2024-10-10T17:43:36Z) - Latent Intuitive Physics: Learning to Transfer Hidden Physics from A 3D Video [58.043569985784806]
We introduce latent intuitive physics, a transfer learning framework for physics simulation.
It can infer hidden properties of fluids from a single 3D video and simulate the observed fluid in novel scenes.
We validate our model in three ways: (i) novel scene simulation with the learned visual-world physics, (ii) future prediction of the observed fluid dynamics, and (iii) supervised particle simulation.
arXiv Detail & Related papers (2024-06-18T16:37:44Z) - A Physics-embedded Deep Learning Framework for Cloth Simulation [6.8806198396336935]
This paper proposes a physics-embedded learning framework that directly encodes physical features of cloth simulation.
The framework can also integrate with other external forces and collision handling through either traditional simulators or sub neural networks.
arXiv Detail & Related papers (2024-03-19T15:21:00Z) - Neural Foundations of Mental Simulation: Future Prediction of Latent
Representations on Dynamic Scenes [3.2744507958793143]
We combine a goal-driven modeling approach with dense neurophysiological data and human behavioral readouts to impinge on this question.
Specifically, we construct and evaluate several classes of sensory-cognitive networks to predict the future state of rich, ethologically-relevant environments.
We find strong differentiation across these model classes in their ability to predict neural and behavioral data both within and across diverse environments.
arXiv Detail & Related papers (2023-05-19T15:56:06Z) - Predictive Experience Replay for Continual Visual Control and
Forecasting [62.06183102362871]
We present a new continual learning approach for visual dynamics modeling and explore its efficacy in visual control and forecasting.
We first propose the mixture world model that learns task-specific dynamics priors with a mixture of Gaussians, and then introduce a new training strategy to overcome catastrophic forgetting.
Our model remarkably outperforms the naive combinations of existing continual learning and visual RL algorithms on DeepMind Control and Meta-World benchmarks with continual visual control tasks.
arXiv Detail & Related papers (2023-03-12T05:08:03Z) - PhysGraph: Physics-Based Integration Using Graph Neural Networks [9.016253794897874]
We focus on the detail enhancement of coarse clothing geometry which has many applications including computer games, virtual reality and virtual try-on.
Our contribution is based on a simple observation: evaluating forces is computationally relatively cheap for traditional simulation methods.
We demonstrate that this idea leads to a learnable module that can be trained on basic internal forces of small mesh patches.
arXiv Detail & Related papers (2023-01-27T16:47:10Z) - Which priors matter? Benchmarking models for learning latent dynamics [70.88999063639146]
Several methods have proposed to integrate priors from classical mechanics into machine learning models.
We take a sober look at the current capabilities of these models.
We find that the use of continuous and time-reversible dynamics benefits models of all classes.
arXiv Detail & Related papers (2021-11-09T23:48:21Z) - Learning Visible Connectivity Dynamics for Cloth Smoothing [17.24004979796887]
We propose to learn a particle-based dynamics model from a partial point cloud observation.
To overcome the challenges of partial observability, we infer which visible points are connected on the underlying cloth mesh.
We show that our method greatly outperforms previous state-of-the-art model-based and model-free reinforcement learning methods in simulation.
arXiv Detail & Related papers (2021-05-21T15:03:29Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.