GeneOH Diffusion: Towards Generalizable Hand-Object Interaction Denoising via Denoising Diffusion

• URL: http://arxiv.org/abs/2402.14810v1
• Date: Thu, 22 Feb 2024 18:59:21 GMT
• Title: GeneOH Diffusion: Towards Generalizable Hand-Object Interaction Denoising via Denoising Diffusion
• Authors: Xueyi Liu, Li Yi
• Abstract summary: Given an erroneous interaction sequence, the objective is to refine the incorrect hand trajectory to remove interaction artifacts for a perceptually realistic sequence. This challenge involves intricate interaction noise, including unnatural hand poses and incorrect hand-object relations. We tackle those challenges through a novel approach, GeneOH Diffusion, incorporating two key designs.
• Score: 27.252526086004956
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: In this work, we tackle the challenging problem of denoising hand-object interactions (HOI). Given an erroneous interaction sequence, the objective is to refine the incorrect hand trajectory to remove interaction artifacts for a perceptually realistic sequence. This challenge involves intricate interaction noise, including unnatural hand poses and incorrect hand-object relations, alongside the necessity for robust generalization to new interactions and diverse noise patterns. We tackle those challenges through a novel approach, GeneOH Diffusion, incorporating two key designs: an innovative contact-centric HOI representation named GeneOH and a new domain-generalizable denoising scheme. The contact-centric representation GeneOH informatively parameterizes the HOI process, facilitating enhanced generalization across various HOI scenarios. The new denoising scheme consists of a canonical denoising model trained to project noisy data samples from a whitened noise space to a clean data manifold and a "denoising via diffusion" strategy which can handle input trajectories with various noise patterns by first diffusing them to align with the whitened noise space and cleaning via the canonical denoiser. Extensive experiments on four benchmarks with significant domain variations demonstrate the superior effectiveness of our method. GeneOH Diffusion also shows promise for various downstream applications. Project website: https://meowuu7.github.io/GeneOH-Diffusion/.

Related papers

• Be Decisive: Noise-Induced Layouts for Multi-Subject Generation [56.80513553424086]
  Complex prompts lead to subject leakage, causing inaccuracies in quantities, attributes, and visual features.<n>We introduce a new approach that predicts a spatial layout aligned with the prompt, derived from the initial noise, and refines it throughout the denoising process.<n>Our method employs a small neural network to predict and refine the evolving noise-induced layout at each denoising step.
  arXiv  Detail & Related papers  (2025-05-27T17:54:24Z)
• On Denoising Walking Videos for Gait Recognition [10.905636016507994]
  We propose DenoisingGait, a novel gait denoising method.<n>Inspired by the philosophy that "what I cannot create, I do not understand", we turn to generative diffusion models.<n>DenoisingGait achieves a new SoTA performance in most cases for both within- and cross-domain evaluations.
  arXiv  Detail & Related papers  (2025-05-24T08:17:34Z)
• Conditional GAN for Enhancing Diffusion Models in Efficient and Authentic Global Gesture Generation from Audios [10.57695963534794]
  Methods based on VAEs are accompanied by issues of local jitter and global instability. We introduce a conditional GAN to capture audio control signals and implicitly match the multimodal denoising distribution between the diffusion and denoising steps.
  arXiv  Detail & Related papers  (2024-10-27T07:25:11Z)
• Denoising as Adaptation: Noise-Space Domain Adaptation for Image Restoration [64.84134880709625]
  We show that it is possible to perform domain adaptation via the noise space using diffusion models. In particular, by leveraging the unique property of how auxiliary conditional inputs influence the multi-step denoising process, we derive a meaningful diffusion loss. We present crucial strategies such as channel-shuffling layer and residual-swapping contrastive learning in the diffusion model.
  arXiv  Detail & Related papers  (2024-06-26T17:40:30Z)
• NoiseDiffusion: Correcting Noise for Image Interpolation with Diffusion Models beyond Spherical Linear Interpolation [86.7260950382448]
  We propose a novel approach to correct noise for image validity, NoiseDiffusion. NoiseDiffusion performs within the noisy image space and injects raw images into these noisy counterparts to address the challenge of information loss.
  arXiv  Detail & Related papers  (2024-03-13T12:32:25Z)
• Listening to the Noise: Blind Denoising with Gibbs Diffusion [4.310554658046964]
  We develop a Gibbs algorithm that alternates sampling steps from a conditional diffusion model trained to map the signal prior to the family of noise distributions. Our theoretical analysis highlights potential pitfalls, guides diagnostic usage, and quantifies errors in the Gibbs stationary distribution. We showcase our method for 1) blind denoising of natural images involving colored noises with unknown amplitude and spectral index, and 2) a cosmology problem, where Bayesian inference of "noise" parameters means constraining models of the evolution of the Universe.
  arXiv  Detail & Related papers  (2024-02-29T18:50:11Z)
• One More Step: A Versatile Plug-and-Play Module for Rectifying Diffusion Schedule Flaws and Enhancing Low-Frequency Controls [77.42510898755037]
  One More Step (OMS) is a compact network that incorporates an additional simple yet effective step during inference. OMS elevates image fidelity and harmonizes the dichotomy between training and inference, while preserving original model parameters. Once trained, various pre-trained diffusion models with the same latent domain can share the same OMS module.
  arXiv  Detail & Related papers  (2023-11-27T12:02:42Z)
• Seismic Data Interpolation via Denoising Diffusion Implicit Models with Coherence-corrected Resampling [7.755439545030289]
  Deep learning models such as U-Net often underperform when the training and test missing patterns do not match. We propose a novel framework that is built upon the multi-modal diffusion models. Inference phase, we introduce the denoising diffusion implicit model to reduce the number of sampling steps. To enhance the coherence and continuity between the revealed traces and the missing traces, we propose two strategies.
  arXiv  Detail & Related papers  (2023-07-09T16:37:47Z)
• DiffusionAD: Norm-guided One-step Denoising Diffusion for Anomaly Detection [80.20339155618612]
  DiffusionAD is a novel anomaly detection pipeline comprising a reconstruction sub-network and a segmentation sub-network.<n>A rapid one-step denoising paradigm achieves hundreds of times acceleration while preserving comparable reconstruction quality.<n>Considering the diversity in the manifestation of anomalies, we propose a norm-guided paradigm to integrate the benefits of multiple noise scales.
  arXiv  Detail & Related papers  (2023-03-15T16:14:06Z)
• Dynamic Dual-Output Diffusion Models [100.32273175423146]
  Iterative denoising-based generation has been shown to be comparable in quality to other classes of generative models. A major drawback of this method is that it requires hundreds of iterations to produce a competitive result. Recent works have proposed solutions that allow for faster generation with fewer iterations, but the image quality gradually deteriorates.
  arXiv  Detail & Related papers  (2022-03-08T11:20:40Z)
• On Procedural Adversarial Noise Attack And Defense [2.5388455804357952]
  adversarial examples would inveigle neural networks to make prediction errors with small per- turbations on the input images. In this paper, we propose two universal adversarial perturbation (UAP) generation methods based on procedural noise functions. Without changing the semantic representations, the adversarial examples generated via our methods show superior performance on the attack.
  arXiv  Detail & Related papers  (2021-08-10T02:47:01Z)
• Dual Adversarial Network: Toward Real-world Noise Removal and Noise Generation [52.75909685172843]
  Real-world image noise removal is a long-standing yet very challenging task in computer vision. We propose a novel unified framework to deal with the noise removal and noise generation tasks. Our method learns the joint distribution of the clean-noisy image pairs.
  arXiv  Detail & Related papers  (2020-07-12T09:16:06Z)
• Learning Model-Blind Temporal Denoisers without Ground Truths [46.778450578529814]
  Denoisers trained with synthetic data often fail to cope with the diversity of unknown noises. Previous image-based method leads to noise overfitting if directly applied to video denoisers. We propose a general framework for video denoising networks that successfully addresses these challenges.
  arXiv  Detail & Related papers  (2020-07-07T07:19:48Z)
• Variational Denoising Network: Toward Blind Noise Modeling and Removal [59.36166491196973]
  Blind image denoising is an important yet very challenging problem in computer vision. We propose a new variational inference method, which integrates both noise estimation and image denoising.
  arXiv  Detail & Related papers  (2019-08-29T15:54:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.