Neural Garment Dynamic Super-Resolution

• URL: http://arxiv.org/abs/2412.06285v1
• Date: Mon, 09 Dec 2024 08:13:38 GMT
• Title: Neural Garment Dynamic Super-Resolution
• Authors: Meng Zhang, Jun Li,
• Abstract summary: We introduce a learning-based method for garment dynamic super-resolution. It is designed to efficiently enhance high-resolution, high-frequency details in low-resolution garment simulations. We demonstrate significant improvements over state-of-the-art alternatives.
• Score: 6.539708325630558
• License:
• Abstract: Achieving efficient, high-fidelity, high-resolution garment simulation is challenging due to its computational demands. Conversely, low-resolution garment simulation is more accessible and ideal for low-budget devices like smartphones. In this paper, we introduce a lightweight, learning-based method for garment dynamic super-resolution, designed to efficiently enhance high-resolution, high-frequency details in low-resolution garment simulations. Starting with low-resolution garment simulation and underlying body motion, we utilize a mesh-graph-net to compute super-resolution features based on coarse garment dynamics and garment-body interactions. These features are then used by a hyper-net to construct an implicit function of detailed wrinkle residuals for each coarse mesh triangle. Considering the influence of coarse garment shapes on detailed wrinkle performance, we correct the coarse garment shape and predict detailed wrinkle residuals using these implicit functions. Finally, we generate detailed high-resolution garment geometry by applying the detailed wrinkle residuals to the corrected coarse garment. Our method enables roll-out prediction by iteratively using its predictions as input for subsequent frames, producing fine-grained wrinkle details to enhance the low-resolution simulation. Despite training on a small dataset, our network robustly generalizes to different body shapes, motions, and garment types not present in the training data. We demonstrate significant improvements over state-of-the-art alternatives, particularly in enhancing the quality of high-frequency, fine-grained wrinkle details.

Related papers

• FitDiT: Advancing the Authentic Garment Details for High-fidelity Virtual Try-on [73.13242624924814]
  Garment perception enhancement technique, FitDiT, is designed for high-fidelity virtual try-on using Diffusion Transformers (DiT) We introduce a garment texture extractor that incorporates garment priors evolution to fine-tune garment feature, facilitating to better capture rich details such as stripes, patterns, and text. We also employ a dilated-relaxed mask strategy that adapts to the correct length of garments, preventing the generation of garments that fill the entire mask area during cross-category try-on.
  arXiv  Detail & Related papers  (2024-11-15T11:02:23Z)
• AnyFit: Controllable Virtual Try-on for Any Combination of Attire Across Any Scenario [50.62711489896909]
  AnyFit surpasses all baselines on high-resolution benchmarks and real-world data by a large gap. AnyFit's impressive performance on high-fidelity virtual try-ons in any scenario from any image, paves a new path for future research within the fashion community.
  arXiv  Detail & Related papers  (2024-05-28T13:33:08Z)
• Towards Loose-Fitting Garment Animation via Generative Model of Deformation Decomposition [4.627632792164547]
  We develop a garment generative model based on deformation decomposition to efficiently simulate loose garment deformation without using linear skinning. We demonstrate our method outperforms state-of-the-art data-driven alternatives through extensive experiments and show qualitative and quantitative analysis of results.
  arXiv  Detail & Related papers  (2023-12-22T11:26:51Z)
• SwinGar: Spectrum-Inspired Neural Dynamic Deformation for Free-Swinging Garments [6.821050909555717]
  We present a spectrum-inspired learning-based approach for generating clothing deformations with dynamic effects and personalized details. Our proposed method overcomes limitations by providing a unified framework that predicts dynamic behavior for different garments. We develop a dynamic clothing deformation estimator that integrates frequency-controllable attention mechanisms with long short-term memory.
  arXiv  Detail & Related papers  (2023-08-05T09:09:50Z)
• MultiScale MeshGraphNets [65.26373813797409]
  We propose two complementary approaches to improve the framework from MeshGraphNets. First, we demonstrate that it is possible to learn accurate surrogate dynamics of a high-resolution system on a much coarser mesh. Second, we introduce a hierarchical approach (MultiScale MeshGraphNets) which passes messages on two different resolutions.
  arXiv  Detail & Related papers  (2022-10-02T20:16:20Z)
• Predicting Loose-Fitting Garment Deformations Using Bone-Driven Motion Networks [63.596602299263935]
  We present a learning algorithm that uses bone-driven motion networks to predict the deformation of loose-fitting garment meshes at interactive rates. We show that our method outperforms state-of-the-art methods in terms of prediction accuracy of mesh deformations by about 20% in RMSE and 10% in Hausdorff distance and STED.
  arXiv  Detail & Related papers  (2022-05-03T07:54:39Z)
• Detail-aware Deep Clothing Animations Infused with Multi-source Attributes [1.6400484152578603]
  This paper presents a novel learning-based clothing deformation method to generate rich and reasonable detailed deformations for garments worn by bodies of various shapes in various animations. In contrast to existing learning-based methods, which require numerous trained models for different garment topologies or poses, we use a unified framework to produce high fidelity deformations efficiently and easily. Experiment results show that our proposed deformation method achieves better performance over existing methods in terms of generalization of ability and quality of details.
  arXiv  Detail & Related papers  (2021-12-15T08:50:49Z)
• Deep Deformation Detail Synthesis for Thin Shell Models [47.442883859643004]
  In physics-based cloth animation, rich folds and detailed wrinkles are achieved at the cost of expensive computational resources and huge labor tuning. We develop a temporally and spatially as-consistent-as-possible deformation representation (named TS-ACAP) and a DeformTransformer network to learn the mapping from low-resolution meshes to detailed ones. Our method is able to produce reliable and realistic animations in various datasets at high frame rates: 10 35 times faster than physics-based simulation, with superior detail synthesis abilities than existing methods.
  arXiv  Detail & Related papers  (2021-02-23T08:09:11Z)
• Interpretable Detail-Fidelity Attention Network for Single Image Super-Resolution [89.1947690981471]
  We propose a purposeful and interpretable detail-fidelity attention network to progressively process smoothes and details in divide-and-conquer manner. Particularly, we propose a Hessian filtering for interpretable feature representation which is high-profile for detail inference. Experiments demonstrate that the proposed methods achieve superior performances over the state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-09-28T08:31:23Z)
• Fully Convolutional Graph Neural Networks for Parametric Virtual Try-On [9.293488420613148]
  We present a learning-based approach for virtual try-on applications based on a fully convolutional graph neural network. In contrast to existing data-driven models, which are trained for a specific garment or mesh topology, our fully convolutional model can cope with a large family of garments. Under the hood, our novel geometric deep learning approach learns to drape 3D garments by decoupling the three different sources of deformations that condition the fit of clothing.
  arXiv  Detail & Related papers  (2020-09-09T22:38:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.