PEMS: Pre-trained Epidemic Time-series Models

• URL: http://arxiv.org/abs/2311.07841v2
• Date: Sun, 19 Nov 2023 19:47:36 GMT
• Title: PEMS: Pre-trained Epidemic Time-series Models
• Authors: Harshavardhan Kamarthi, B. Aditya Prakash
• Abstract summary: We introduce Pre-trained Epidemic Time-Series Models (PEMS) PEMS learn from diverse time-series datasets of a variety of diseases by formulating pre-training as a set of self-supervised learning (SSL) tasks. The resultant PEM outperforms previous state-of-the-art methods in various downstream time-series tasks across datasets of varying seasonal patterns, geography, and mechanism of contagion including the novel Covid-19 pandemic unseen in pre-trained data with better efficiency using smaller fraction of datasets.
• Score: 23.897701882327972
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Providing accurate and reliable predictions about the future of an epidemic is an important problem for enabling informed public health decisions. Recent works have shown that leveraging data-driven solutions that utilize advances in deep learning methods to learn from past data of an epidemic often outperform traditional mechanistic models. However, in many cases, the past data is sparse and may not sufficiently capture the underlying dynamics. While there exists a large amount of data from past epidemics, leveraging prior knowledge from time-series data of other diseases is a non-trivial challenge. Motivated by the success of pre-trained models in language and vision tasks, we tackle the problem of pre-training epidemic time-series models to learn from multiple datasets from different diseases and epidemics. We introduce Pre-trained Epidemic Time-Series Models (PEMS) that learn from diverse time-series datasets of a variety of diseases by formulating pre-training as a set of self-supervised learning (SSL) tasks. We tackle various important challenges specific to pre-training for epidemic time-series such as dealing with heterogeneous dynamics and efficiently capturing useful patterns from multiple epidemic datasets by carefully designing the SSL tasks to learn important priors about the epidemic dynamics that can be leveraged for fine-tuning to multiple downstream tasks. The resultant PEM outperforms previous state-of-the-art methods in various downstream time-series tasks across datasets of varying seasonal patterns, geography, and mechanism of contagion including the novel Covid-19 pandemic unseen in pre-trained data with better efficiency using smaller fraction of datasets.

Related papers

• Membership Inference Attacks Against Time-Series Models [0.8437187555622164]
  Time-series data that contains personal information, particularly in the medical field, presents serious privacy concerns. We explore existing techniques on time-series models, and introduce new features, focusing on seasonality. Our results enhance the effectiveness of MIAs in identifying membership, improving the understanding of privacy risks in medical data applications.
  arXiv  Detail & Related papers  (2024-07-03T07:34:49Z)
• Advancing Real-time Pandemic Forecasting Using Large Language Models: A COVID-19 Case Study [39.70947911556511]
  Existing forecasting models struggle with the multifaceted nature of relevant data and robust results translation. Our work introduces PandemicLLM, a novel framework that reformulates real-time forecasting of disease spread as a text reasoning problem. The model is applied to the COVID-19 pandemic, and trained to utilize textual public health policies, genomic surveillance, spatial, and epidemiological time series data.
  arXiv  Detail & Related papers  (2024-04-10T12:22:03Z)
• Data-Centric Epidemic Forecasting: A Survey [56.99209141838794]
  This survey delves into various data-driven methodological and practical advancements. We enumerate the large number of epidemiological datasets and novel data streams that are relevant to epidemic forecasting. We also discuss experiences and challenges that arise in real-world deployment of these forecasting systems.
  arXiv  Detail & Related papers  (2022-07-19T16:15:11Z)
• LifeLonger: A Benchmark for Continual Disease Classification [59.13735398630546]
  We introduce LifeLonger, a benchmark for continual disease classification on the MedMNIST collection. Task and class incremental learning of diseases address the issue of classifying new samples without re-training the models from scratch. Cross-domain incremental learning addresses the issue of dealing with datasets originating from different institutions while retaining the previously obtained knowledge.
  arXiv  Detail & Related papers  (2022-04-12T12:25:05Z)
• Unsupervised Pre-Training on Patient Population Graphs for Patient-Level Predictions [48.02011627390706]
  Pre-training has shown success in different areas of machine learning, such as Computer Vision (CV), Natural Language Processing (NLP) and medical imaging. In this paper, we apply unsupervised pre-training to heterogeneous, multi-modal EHR data for patient outcome prediction. We find that our proposed graph based pre-training method helps in modeling the data at a population level.
  arXiv  Detail & Related papers  (2022-03-23T17:59:45Z)
• Practical Challenges in Differentially-Private Federated Survival Analysis of Medical Data [57.19441629270029]
  In this paper, we take advantage of the inherent properties of neural networks to federate the process of training of survival analysis models. In the realistic setting of small medical datasets and only a few data centers, this noise makes it harder for the models to converge. We propose DPFed-post which adds a post-processing stage to the private federated learning scheme.
  arXiv  Detail & Related papers  (2022-02-08T10:03:24Z)
• Pre-training transformer-based framework on large-scale pediatric claims data for downstream population-specific tasks [3.1580072841682734]
  This study presents the Claim Pre-Training (Claim-PT) framework, a generic pre-training model that first trains on the entire pediatric claims dataset. The effective knowledge transfer is completed through the task-aware fine-tuning stage. We conducted experiments on a real-world claims dataset with more than one million patient records.
  arXiv  Detail & Related papers  (2021-06-24T15:25:41Z)
• An Optimal Control Approach to Learning in SIDARTHE Epidemic model [67.22168759751541]
  We propose a general approach for learning time-variant parameters of dynamic compartmental models from epidemic data. We forecast the epidemic evolution in Italy and France.
  arXiv  Detail & Related papers  (2020-10-28T10:58:59Z)
• Select-ProtoNet: Learning to Select for Few-Shot Disease Subtype Prediction [55.94378672172967]
  We focus on few-shot disease subtype prediction problem, identifying subgroups of similar patients. We introduce meta learning techniques to develop a new model, which can extract the common experience or knowledge from interrelated clinical tasks. Our new model is built upon a carefully designed meta-learner, called Prototypical Network, that is a simple yet effective meta learning machine for few-shot image classification.
  arXiv  Detail & Related papers  (2020-09-02T02:50:30Z)
• Multi-task Learning via Adaptation to Similar Tasks for Mortality Prediction of Diverse Rare Diseases [10.020413101958944]
  Mortality prediction of diverse rare diseases using electronic health record (EHR) data is a crucial task for intelligent healthcare. Data insufficiency and the clinical diversity of rare diseases make it hard for directly training deep learning models on individual disease data. We use Ada-Sit to train long short-term memory networks (LSTM) based prediction models on longitudinal EHR data.
  arXiv  Detail & Related papers  (2020-04-11T06:15:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.