MANet: Improving Video Denoising with a Multi-Alignment Network

• URL: http://arxiv.org/abs/2202.09704v1
• Date: Sun, 20 Feb 2022 00:52:07 GMT
• Title: MANet: Improving Video Denoising with a Multi-Alignment Network
• Authors: Yaping Zhao, Haitian Zheng, Zhongrui Wang, Jiebo Luo, Edmund Y. Lam
• Abstract summary: We present a multi-alignment network, which generates multiple flow proposals followed by attention-based averaging. Experiments on a large-scale video dataset demonstrate that our method improves the denoising baseline model by 0.2dB.
• Score: 72.93429911044903
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In video denoising, the adjacent frames often provide very useful information, but accurate alignment is needed before such information can be harnassed. In this work, we present a multi-alignment network, which generates multiple flow proposals followed by attention-based averaging. It serves to mimics the non-local mechanism, suppressing noise by averaging multiple observations. Our approach can be applied to various state-of-the-art models that are based on flow estimation. Experiments on a large-scale video dataset demonstrate that our method improves the denoising baseline model by 0.2dB, and further reduces the parameters by 47% with model distillation.

Related papers

• MULDE: Multiscale Log-Density Estimation via Denoising Score Matching for Video Anomaly Detection [15.72443573134312]
  We treat feature vectors extracted from videos as realizations of a random variable with a fixed distribution. We train our video anomaly detector using a modification of denoising score matching. Our experiments on five popular video anomaly detection benchmarks demonstrate state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-03-21T15:46:19Z)
• Blue noise for diffusion models [50.99852321110366]
  We introduce a novel and general class of diffusion models taking correlated noise within and across images into account. Our framework allows introducing correlation across images within a single mini-batch to improve gradient flow. We perform both qualitative and quantitative evaluations on a variety of datasets using our method.
  arXiv  Detail & Related papers  (2024-02-07T14:59:25Z)
• ExposureDiffusion: Learning to Expose for Low-light Image Enhancement [87.08496758469835]
  This work addresses the issue by seamlessly integrating a diffusion model with a physics-based exposure model. Our method obtains significantly improved performance and reduced inference time compared with vanilla diffusion models. The proposed framework can work with both real-paired datasets, SOTA noise models, and different backbone networks.
  arXiv  Detail & Related papers  (2023-07-15T04:48:35Z)
• VideoFusion: Decomposed Diffusion Models for High-Quality Video Generation [88.49030739715701]
  This work presents a decomposed diffusion process via resolving the per-frame noise into a base noise that is shared among all frames and a residual noise that varies along the time axis. Experiments on various datasets confirm that our approach, termed as VideoFusion, surpasses both GAN-based and diffusion-based alternatives in high-quality video generation.
  arXiv  Detail & Related papers  (2023-03-15T02:16:39Z)
• Enhancing convolutional neural network generalizability via low-rank weight approximation [6.763245393373041]
  Sufficient denoising is often an important first step for image processing. Deep neural networks (DNNs) have been widely used for image denoising. We introduce a new self-supervised framework for image denoising based on the Tucker low-rank tensor approximation.
  arXiv  Detail & Related papers  (2022-09-26T14:11:05Z)
• Learning Task-Oriented Flows to Mutually Guide Feature Alignment in Synthesized and Real Video Denoising [137.5080784570804]
  Video denoising aims at removing noise from videos to recover clean ones. Some existing works show that optical flow can help the denoising by exploiting the additional spatial-temporal clues from nearby frames. We propose a new multi-scale refined optical flow-guided video denoising method, which is more robust to different noise levels.
  arXiv  Detail & Related papers  (2022-08-25T00:09:18Z)
• PD-Flow: A Point Cloud Denoising Framework with Normalizing Flows [20.382995180671205]
  Point cloud denoising aims to restore clean point clouds from raw observations corrupted by noise and outliers. We present a novel deep learning-based denoising model, that incorporates normalizing flows and noise disentanglement techniques.
  arXiv  Detail & Related papers  (2022-03-11T14:17:58Z)
• Diffusion-Based Representation Learning [65.55681678004038]
  We augment the denoising score matching framework to enable representation learning without any supervised signal. In contrast, the introduced diffusion-based representation learning relies on a new formulation of the denoising score matching objective. Using the same approach, we propose to learn an infinite-dimensional latent code that achieves improvements of state-of-the-art models on semi-supervised image classification.
  arXiv  Detail & Related papers  (2021-05-29T09:26:02Z)
• Robust Unsupervised Multi-Object Tracking in Noisy Environments [5.409476600348953]
  We introduce a robust unsupervised multi-object tracking (MOT) model: AttU-Net. The proposed single-head attention model helps limit the negative impact of noise by learning visual representations at different segment scales. We evaluate our method in the MNIST and the Atari game video benchmark.
  arXiv  Detail & Related papers  (2021-05-20T19:38:03Z)
• Learning Model-Blind Temporal Denoisers without Ground Truths [46.778450578529814]
  Denoisers trained with synthetic data often fail to cope with the diversity of unknown noises. Previous image-based method leads to noise overfitting if directly applied to video denoisers. We propose a general framework for video denoising networks that successfully addresses these challenges.
  arXiv  Detail & Related papers  (2020-07-07T07:19:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.