Spectroscopic data de-noising via training-set-free deep learning method

• URL: http://arxiv.org/abs/2210.10494v2
• Date: Mon, 15 May 2023 12:07:55 GMT
• Title: Spectroscopic data de-noising via training-set-free deep learning method
• Authors: Dongchen Huang, Junde Liu, Tian Qian, and Yi-feng Yang
• Abstract summary: We develop a de-noising method for extracting intrinsic spectral information without the need for a training set. This is possible as our method leverages the self-correlation information of the spectra themselves. It preserves the intrinsic energy band features and thus facilitates further analysis and processing.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: De-noising plays a crucial role in the post-processing of spectra. Machine learning-based methods show good performance in extracting intrinsic information from noisy data, but often require a high-quality training set that is typically inaccessible in real experimental measurements. Here, using spectra in angle-resolved photoemission spectroscopy (ARPES) as an example, we develop a de-noising method for extracting intrinsic spectral information without the need for a training set. This is possible as our method leverages the self-correlation information of the spectra themselves. It preserves the intrinsic energy band features and thus facilitates further analysis and processing. Moreover, since our method is not limited by specific properties of the training set compared to previous ones, it may well be extended to other fields and application scenarios where obtaining high-quality multidimensional training data is challenging.

Related papers

• Datacube segmentation via Deep Spectral Clustering [76.48544221010424]
  Extended Vision techniques often pose a challenge in their interpretation. The huge dimensionality of data cube spectra poses a complex task in its statistical interpretation. In this paper, we explore the possibility of applying unsupervised clustering methods in encoded space. A statistical dimensional reduction is performed by an ad hoc trained (Variational) AutoEncoder, while the clustering process is performed by a (learnable) iterative K-Means clustering algorithm.
  arXiv  Detail & Related papers  (2024-01-31T09:31:28Z)
• Hodge-Aware Contrastive Learning [101.56637264703058]
  Simplicial complexes prove effective in modeling data with multiway dependencies. We develop a contrastive self-supervised learning approach for processing simplicial data.
  arXiv  Detail & Related papers  (2023-09-14T00:40:07Z)
• Spectrum-BERT: Pre-training of Deep Bidirectional Transformers for Spectral Classification of Chinese Liquors [0.0]
  We propose a pre-training method of deep bidirectional transformers for spectral classification of Chinese liquors, abbreviated as Spectrum-BERT. We elaborately design two pre-training tasks, Next Curve Prediction (NCP) and Masked Curve Model (MCM), so that the model can effectively utilize unlabeled samples. In the comparative experiments, the proposed Spectrum-BERT significantly outperforms the baselines in multiple metrics.
  arXiv  Detail & Related papers  (2022-10-22T13:11:25Z)
• Spectral Decomposition Representation for Reinforcement Learning [100.0424588013549]
  We propose an alternative spectral method, Spectral Decomposition Representation (SPEDER), that extracts a state-action abstraction from the dynamics without inducing spurious dependence on the data collection policy. A theoretical analysis establishes the sample efficiency of the proposed algorithm in both the online and offline settings. An experimental investigation demonstrates superior performance over current state-of-the-art algorithms across several benchmarks.
  arXiv  Detail & Related papers  (2022-08-19T19:01:30Z)
• Toward deep-learning-assisted spectrally-resolved imaging of magnetic noise [52.77024349608834]
  We implement a deep neural network to efficiently reconstruct the spectral density of the underlying fluctuating magnetic field. These results create opportunities for the application of machine-learning methods to color-center-based nanoscale sensing and imaging.
  arXiv  Detail & Related papers  (2022-08-01T19:18:26Z)
• Don't Pay Attention to the Noise: Learning Self-supervised Representations of Light Curves with a Denoising Time Series Transformer [0.0]
  We propose a simple Transformer model -- called Denoising Time Series Transformer (DTST) We show that it excels at removing the noise and outliers in datasets of time series when trained with a masked objective. We present experiments on real stellar light curves from the Transiting Exoplanet Space Satellite (TESS)
  arXiv  Detail & Related papers  (2022-07-06T16:10:11Z)
• Explainable Predictive Modeling for Limited Spectral Data [0.0]
  We introduce applying recent explainable AI techniques to interpret the prediction outcomes of high-dimensional and limited spectral data. Due to instrument resolution limitations, pinpointing important regions of the spectroscopy data creates a pathway to optimize the data collection process. We specifically design three different scenarios to ensure that the evaluation of ML models is robust for the real-time practice.
  arXiv  Detail & Related papers  (2022-02-09T15:46:17Z)
• Deep learning-based statistical noise reduction for multidimensional spectral data [3.0396858935319724]
  We demonstrate a denoising method that utilizes deep learning as an intelligent way to overcome the constraint. We show that the denoising neural network allows us to perform similar level of second-derivative and line shape analysis on data taken with two orders of magnitude less acquisition time.
  arXiv  Detail & Related papers  (2021-07-02T05:37:16Z)
• Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain [88.7339322596758]
  We present a novel Spatial-Phase Shallow Learning (SPSL) method, which combines spatial image and phase spectrum to capture the up-sampling artifacts of face forgery. SPSL can achieve the state-of-the-art performance on cross-datasets evaluation as well as multi-class classification and obtain comparable results on single dataset evaluation.
  arXiv  Detail & Related papers  (2021-03-02T16:45:08Z)
• Spectral Methods for Data Science: A Statistical Perspective [37.2486912080998]
  Spectral methods have emerged as a simple yet surprisingly effective approach for extracting information from massive, noisy and incomplete data. This book aims to present a systematic, comprehensive, yet accessible introduction to spectral methods from a modern statistical perspective.
  arXiv  Detail & Related papers  (2020-12-15T18:40:56Z)
• Spectral Analysis Network for Deep Representation Learning and Image Clustering [53.415803942270685]
  This paper proposes a new network structure for unsupervised deep representation learning based on spectral analysis. It can identify the local similarities among images in patch level and thus more robust against occlusion. It can learn more clustering-friendly representations and is capable to reveal the deep correlations among data samples.
  arXiv  Detail & Related papers  (2020-09-11T05:07:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.