Don't Pay Attention to the Noise: Learning Self-supervised Representations of Light Curves with a Denoising Time Series Transformer

• URL: http://arxiv.org/abs/2207.02777v1
• Date: Wed, 6 Jul 2022 16:10:11 GMT
• Title: Don't Pay Attention to the Noise: Learning Self-supervised Representations of Light Curves with a Denoising Time Series Transformer
• Authors: Mario Morvan, Nikolaos Nikolaou, Kai Hou Yip, Ingo Waldmann
• Abstract summary: We propose a simple Transformer model -- called Denoising Time Series Transformer (DTST) We show that it excels at removing the noise and outliers in datasets of time series when trained with a masked objective. We present experiments on real stellar light curves from the Transiting Exoplanet Space Satellite (TESS)
• Score: 0.0
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Astrophysical light curves are particularly challenging data objects due to the intensity and variety of noise contaminating them. Yet, despite the astronomical volumes of light curves available, the majority of algorithms used to process them are still operating on a per-sample basis. To remedy this, we propose a simple Transformer model -- called Denoising Time Series Transformer (DTST) -- and show that it excels at removing the noise and outliers in datasets of time series when trained with a masked objective, even when no clean targets are available. Moreover, the use of self-attention enables rich and illustrative queries into the learned representations. We present experiments on real stellar light curves from the Transiting Exoplanet Space Satellite (TESS), showing advantages of our approach compared to traditional denoising techniques.

Related papers

• Transfer Learning Adapts to Changing PSD in Gravitational Wave Data [0.0]
  Noise in gravitational wave data poses major challenges in signal identification. Traditional noise suppression methods often fall short in fully addressing the non-Gaussian effects in the data. We develop a novel training methodology that enables it to accurately detect gravitational waves amidst highly complex noise.
  arXiv  Detail & Related papers  (2024-10-15T05:27:03Z)
• Self-supervised learning for denoising quasiparticle interference data [0.0]
  Tunneling spectroscopy is an important tool for the study of correlated electron systems. Machine learning provides techniques to reduce the noise in post-processing. We adapt the unsupervised Noise2Noise and self-supervised Noise2Self algorithms, which allow for denoising without clean examples.
  arXiv  Detail & Related papers  (2024-09-13T15:06:51Z)
• Realistic Noise Synthesis with Diffusion Models [68.48859665320828]
  Deep image denoising models often rely on large amount of training data for the high quality performance. We propose a novel method that synthesizes realistic noise using diffusion models, namely Realistic Noise Synthesize Diffusor (RNSD) RNSD can incorporate guided multiscale content, such as more realistic noise with spatial correlations can be generated at multiple frequencies.
  arXiv  Detail & Related papers  (2023-05-23T12:56:01Z)
• Advancing Unsupervised Low-light Image Enhancement: Noise Estimation, Illumination Interpolation, and Self-Regulation [55.07472635587852]
  Low-Light Image Enhancement (LLIE) techniques have made notable advancements in preserving image details and enhancing contrast. These approaches encounter persistent challenges in efficiently mitigating dynamic noise and accommodating diverse low-light scenarios. We first propose a method for estimating the noise level in low light images in a quick and accurate way. We then devise a Learnable Illumination Interpolator (LII) to satisfy general constraints between illumination and input.
  arXiv  Detail & Related papers  (2023-05-17T13:56:48Z)
• Representing Noisy Image Without Denoising [91.73819173191076]
  Fractional-order Moments in Radon space (FMR) is designed to derive robust representation directly from noisy images. Unlike earlier integer-order methods, our work is a more generic design taking such classical methods as special cases.
  arXiv  Detail & Related papers  (2023-01-18T10:13:29Z)
• Spectroscopic data de-noising via training-set-free deep learning method [0.0]
  We develop a de-noising method for extracting intrinsic spectral information without the need for a training set. This is possible as our method leverages the self-correlation information of the spectra themselves. It preserves the intrinsic energy band features and thus facilitates further analysis and processing.
  arXiv  Detail & Related papers  (2022-10-19T12:04:35Z)
• Physics-based Noise Modeling for Extreme Low-light Photography [63.65570751728917]
  We study the noise statistics in the imaging pipeline of CMOS photosensors. We formulate a comprehensive noise model that can accurately characterize the real noise structures. Our noise model can be used to synthesize realistic training data for learning-based low-light denoising algorithms.
  arXiv  Detail & Related papers  (2021-08-04T16:36:29Z)
• Deep learning-based statistical noise reduction for multidimensional spectral data [3.0396858935319724]
  We demonstrate a denoising method that utilizes deep learning as an intelligent way to overcome the constraint. We show that the denoising neural network allows us to perform similar level of second-derivative and line shape analysis on data taken with two orders of magnitude less acquisition time.
  arXiv  Detail & Related papers  (2021-07-02T05:37:16Z)
• Improving Blind Spot Denoising for Microscopy [73.94017852757413]
  We present a novel way to improve the quality of self-supervised denoising. We assume the clean image to be the result of a convolution with a point spread function (PSF) and explicitly include this operation at the end of our neural network.
  arXiv  Detail & Related papers  (2020-08-19T13:06:24Z)
• Learning Model-Blind Temporal Denoisers without Ground Truths [46.778450578529814]
  Denoisers trained with synthetic data often fail to cope with the diversity of unknown noises. Previous image-based method leads to noise overfitting if directly applied to video denoisers. We propose a general framework for video denoising networks that successfully addresses these challenges.
  arXiv  Detail & Related papers  (2020-07-07T07:19:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.