Architectural Optimization and Feature Learning for High-Dimensional Time Series Datasets

• URL: http://arxiv.org/abs/2202.13486v1
• Date: Sun, 27 Feb 2022 23:41:23 GMT
• Title: Architectural Optimization and Feature Learning for High-Dimensional Time Series Datasets
• Authors: Robert E. Colgan, Jingkai Yan, Zsuzsa M\'arka, Imre Bartos, Szabolcs M\'arka, and John N. Wright
• Abstract summary: We study the problem of predicting the presence of transient noise artifacts in a gravitational wave detector. We introduce models that reduce the error rate by over 60% compared to the previous state of the art.
• Score: 0.7388859384645262
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As our ability to sense increases, we are experiencing a transition from data-poor problems, in which the central issue is a lack of relevant data, to data-rich problems, in which the central issue is to identify a few relevant features in a sea of observations. Motivated by applications in gravitational-wave astrophysics, we study the problem of predicting the presence of transient noise artifacts in a gravitational wave detector from a rich collection of measurements from the detector and its environment. We argue that feature learning--in which relevant features are optimized from data--is critical to achieving high accuracy. We introduce models that reduce the error rate by over 60\% compared to the previous state of the art, which used fixed, hand-crafted features. Feature learning is useful not only because it improves performance on prediction tasks; the results provide valuable information about patterns associated with phenomena of interest that would otherwise be undiscoverable. In our application, features found to be associated with transient noise provide diagnostic information about its origin and suggest mitigation strategies. Learning in high-dimensional settings is challenging. Through experiments with a variety of architectures, we identify two key factors in successful models: sparsity, for selecting relevant variables within the high-dimensional observations; and depth, which confers flexibility for handling complex interactions and robustness with respect to temporal variations. We illustrate their significance through systematic experiments on real detector data. Our results provide experimental corroboration of common assumptions in the machine-learning community and have direct applicability to improving our ability to sense gravitational waves, as well as to many other problem settings with similarly high-dimensional, noisy, or partly irrelevant data.

Related papers

• Impact of Label Noise on Learning Complex Features [0.5249805590164901]
  We show that pretraining promotes learning complex functions and diverse features in the presence of noise. Our experiments demonstrate that pre-training with noisy labels encourages gradient descent to find alternate minima.
  arXiv  Detail & Related papers  (2024-11-07T09:47:18Z)
• Localized Gaussians as Self-Attention Weights for Point Clouds Correspondence [92.07601770031236]
  We investigate semantically meaningful patterns in the attention heads of an encoder-only Transformer architecture. We find that fixing the attention weights not only accelerates the training process but also enhances the stability of the optimization.
  arXiv  Detail & Related papers  (2024-09-20T07:41:47Z)
• Machine Learning and Feature Ranking for Impact Fall Detection Event Using Multisensor Data [1.9731252964716424]
  We employ a feature selection process to identify the most relevant features from the multisensor UP-FALL dataset. We then evaluate the efficiency of various machine learning models in detecting the impact moment. Our results achieve high accuracy rates in impact detection, showcasing the power of leveraging multisensor data for fall detection tasks.
  arXiv  Detail & Related papers  (2023-12-21T01:05:44Z)
• Deep networks for system identification: a Survey [56.34005280792013]
  System identification learns mathematical descriptions of dynamic systems from input-output data. Main aim of the identified model is to predict new data from previous observations. We discuss architectures commonly adopted in the literature, like feedforward, convolutional, and recurrent networks.
  arXiv  Detail & Related papers  (2023-01-30T12:38:31Z)
• Anomaly detection using data depth: multivariate case [3.046315755726937]
  Anomaly detection is a branch of data analysis and machine learning. Data depth is a statistical function that measures belongingness of any point of the space to a data set. This article studies data depth as an efficient anomaly detection tool, assigning abnormality labels to observations with lower depth values.
  arXiv  Detail & Related papers  (2022-10-06T12:14:25Z)
• Information-Theoretic Odometry Learning [83.36195426897768]
  We propose a unified information theoretic framework for learning-motivated methods aimed at odometry estimation. The proposed framework provides an elegant tool for performance evaluation and understanding in information-theoretic language.
  arXiv  Detail & Related papers  (2022-03-11T02:37:35Z)
• Audacity of huge: overcoming challenges of data scarcity and data quality for machine learning in computational materials discovery [1.0036312061637764]
  Machine learning (ML)-accelerated discovery requires large amounts of high-fidelity data to reveal predictive structure-property relationships. For many properties of interest in materials discovery, the challenging nature and high cost of data generation has resulted in a data landscape that is scarcely populated and of dubious quality. In the absence of manual curation, increasingly sophisticated natural language processing and automated image analysis are making it possible to learn structure-property relationships from the literature.
  arXiv  Detail & Related papers  (2021-11-02T21:43:58Z)
• Learning Neural Causal Models with Active Interventions [83.44636110899742]
  We introduce an active intervention-targeting mechanism which enables a quick identification of the underlying causal structure of the data-generating process. Our method significantly reduces the required number of interactions compared with random intervention targeting. We demonstrate superior performance on multiple benchmarks from simulated to real-world data.
  arXiv  Detail & Related papers  (2021-09-06T13:10:37Z)
• Towards Unbiased Visual Emotion Recognition via Causal Intervention [63.74095927462]
  We propose a novel Emotion Recognition Network (IERN) to alleviate the negative effects brought by the dataset bias. A series of designed tests validate the effectiveness of IERN, and experiments on three emotion benchmarks demonstrate that IERN outperforms other state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-07-26T10:40:59Z)
• Influence Functions in Deep Learning Are Fragile [52.31375893260445]
  influence functions approximate the effect of samples in test-time predictions. influence estimates are fairly accurate for shallow networks. Hessian regularization is important to get highquality influence estimates.
  arXiv  Detail & Related papers  (2020-06-25T18:25:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.