TraM : Enhancing User Sleep Prediction with Transformer-based Multivariate Time Series Modeling and Machine Learning Ensembles

• URL: http://arxiv.org/abs/2410.11293v1
• Date: Tue, 15 Oct 2024 05:29:55 GMT
• Title: TraM : Enhancing User Sleep Prediction with Transformer-based Multivariate Time Series Modeling and Machine Learning Ensembles
• Authors: Jinjae Kim, Minjeong Ma, Eunjee Choi, Keunhee Cho, Chanwoo Lee,
• Abstract summary: This paper presents a novel approach that leverages Transformer-based multivariate time series model and Machine Learning Ensembles to predict the quality of human sleep, emotional states, and stress levels. Time Series Transformer was used for labels where time series characteristics are crucial, while Machine Learning Ensembles were employed for labels requiring comprehensive daily activity statistics. The proposed model, TraM, scored 6.10 out of 10 in experiments, demonstrating superior performance compared to other methodologies.
• Score: 3.565151496245487
• License:
• Abstract: This paper presents a novel approach that leverages Transformer-based multivariate time series model and Machine Learning Ensembles to predict the quality of human sleep, emotional states, and stress levels. A formula to calculate the labels was developed, and the various models were applied to user data. Time Series Transformer was used for labels where time series characteristics are crucial, while Machine Learning Ensembles were employed for labels requiring comprehensive daily activity statistics. Time Series Transformer excels in capturing the characteristics of time series through pre-training, while Machine Learning Ensembles select machine learning models that meet our categorization criteria. The proposed model, TraM, scored 6.10 out of 10 in experiments, demonstrating superior performance compared to other methodologies. The code and configuration for the TraM framework are available at: https://github.com/jin-jae/ETRI-Paper-Contest.

Related papers

• Metadata Matters for Time Series: Informative Forecasting with Transformers [70.38241681764738]
  We propose a Metadata-informed Time Series Transformer (MetaTST) for time series forecasting. To tackle the unstructured nature of metadata, MetaTST formalizes them into natural languages by pre-designed templates. A Transformer encoder is employed to communicate series and metadata tokens, which can extend series representations by metadata information.
  arXiv  Detail & Related papers  (2024-10-04T11:37:55Z)
• UniTST: Effectively Modeling Inter-Series and Intra-Series Dependencies for Multivariate Time Series Forecasting [98.12558945781693]
  We propose a transformer-based model UniTST containing a unified attention mechanism on the flattened patch tokens. Although our proposed model employs a simple architecture, it offers compelling performance as shown in our experiments on several datasets for time series forecasting.
  arXiv  Detail & Related papers  (2024-06-07T14:39:28Z)
• UNITS: A Unified Multi-Task Time Series Model [31.675845788410246]
  We introduce UniTS, a multi-task time series model that uses task tokenization to express predictive and generative tasks within a single model. Across 38 datasets spanning human activity sensors, healthcare, engineering, and finance domains, UniTS model performs favorably against 12 forecasting models, 20 classification models, 18 anomaly detection models, and 16 imputation models.
  arXiv  Detail & Related papers  (2024-02-29T21:25:58Z)
• Unified Training of Universal Time Series Forecasting Transformers [104.56318980466742]
  We present a Masked-based Universal Time Series Forecasting Transformer (Moirai) Moirai is trained on our newly introduced Large-scale Open Time Series Archive (LOTSA) featuring over 27B observations across nine domains. Moirai achieves competitive or superior performance as a zero-shot forecaster when compared to full-shot models.
  arXiv  Detail & Related papers  (2024-02-04T20:00:45Z)
• Timer: Generative Pre-trained Transformers Are Large Time Series Models [83.03091523806668]
  This paper aims at the early development of large time series models (LTSM) During pre-training, we curate large-scale datasets with up to 1 billion time points. To meet diverse application needs, we convert forecasting, imputation, and anomaly detection of time series into a unified generative task.
  arXiv  Detail & Related papers  (2024-02-04T06:55:55Z)
• TACTiS-2: Better, Faster, Simpler Attentional Copulas for Multivariate Time Series [57.4208255711412]
  Building on copula theory, we propose a simplified objective for the recently-introduced transformer-based attentional copulas (TACTiS) We show that the resulting model has significantly better training dynamics and achieves state-of-the-art performance across diverse real-world forecasting tasks.
  arXiv  Detail & Related papers  (2023-10-02T16:45:19Z)
• A Transformer-based Framework For Multi-variate Time Series: A Remaining Useful Life Prediction Use Case [4.0466311968093365]
  This work proposed an encoder-transformer architecture-based framework for time series prediction. We validated the effectiveness of the proposed framework on all four sets of the C-MAPPS benchmark dataset. To enable the model awareness of the initial stages of the machine life and its degradation path, a novel expanding window method was proposed.
  arXiv  Detail & Related papers  (2023-08-19T02:30:35Z)
• Ti-MAE: Self-Supervised Masked Time Series Autoencoders [16.98069693152999]
  We propose a novel framework named Ti-MAE, in which the input time series are assumed to follow an integrate distribution. Ti-MAE randomly masks out embedded time series data and learns an autoencoder to reconstruct them at the point-level. Experiments on several public real-world datasets demonstrate that our framework of masked autoencoding could learn strong representations directly from the raw data.
  arXiv  Detail & Related papers  (2023-01-21T03:20:23Z)
• Merlion: A Machine Learning Library for Time Series [73.46386700728577]
  Merlion is an open-source machine learning library for time series. It features a unified interface for models and datasets for anomaly detection and forecasting. Merlion also provides a unique evaluation framework that simulates the live deployment and re-training of a model in production.
  arXiv  Detail & Related papers  (2021-09-20T02:03:43Z)
• Deep Transformer Models for Time Series Forecasting: The Influenza Prevalence Case [2.997238772148965]
  Time series data are prevalent in many scientific and engineering disciplines. We present a new approach to time series forecasting using Transformer-based machine learning models. We show that the forecasting results produced by our approach are favorably comparable to the state-of-the-art.
  arXiv  Detail & Related papers  (2020-01-23T00:22:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.