MOTOR: A Time-To-Event Foundation Model For Structured Medical Records

• URL: http://arxiv.org/abs/2301.03150v4
• Date: Tue, 5 Dec 2023 00:24:44 GMT
• Title: MOTOR: A Time-To-Event Foundation Model For Structured Medical Records
• Abstract summary: We present a self-supervised, time-to-event (TTE) foundation model called MOTOR (Many Outcome Time Oriented Representations) TTE models are used for estimating the probability distribution of the time until a specific event occurs. Motor addresses this challenge by pretraining on up to 55M patient records (9B clinical events)
• Score: 4.773344142823636
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a self-supervised, time-to-event (TTE) foundation model called MOTOR (Many Outcome Time Oriented Representations) which is pretrained on timestamped sequences of events in electronic health records (EHR) and health insurance claims. TTE models are used for estimating the probability distribution of the time until a specific event occurs, which is an important task in medical settings. TTE models provide many advantages over classification using fixed time horizons, including naturally handling censored observations, but are challenging to train with limited labeled data. MOTOR addresses this challenge by pretraining on up to 55M patient records (9B clinical events). We evaluate MOTOR's transfer learning performance on 19 tasks, across 3 patient databases (a private EHR system, MIMIC-IV, and Merative claims data). Task-specific models adapted from MOTOR improve time-dependent C statistics by 4.6% over state-of-the-art, improve label efficiency by up to 95% ,and are more robust to temporal distributional shifts. We further evaluate cross-site portability by adapting our MOTOR foundation model for six prediction tasks on the MIMIC-IV dataset, where it outperforms all baselines. MOTOR is the first foundation model for medical TTE predictions and we release a 143M parameter pretrained model for research use at [redacted URL].

Related papers

• Large Language Model-Powered Query-Driven Event Timeline Summarization in Industrial Search [49.02761046363752]
  We present QDET (Query-Driven Event Timeline Summarization), a production system deployed on Baidu Search.<n>QDET identifies and organizes sub-events closely relevant to the query from noisy candidate sets formed by millions of documents retrieved daily.<n>Online A/B tests on Baidu Search validate real-world effectiveness, showing 5.5% CTR improvement, 4.6% longer dwell time, and 4.4% deeper exploration.
  arXiv  Detail & Related papers  (2026-05-26T14:16:27Z)
• Towards Brain MRI Foundation Models for the Clinic: Findings from the FOMO25 Challenge [33.95846433611413]
  We organized the FOMO25 challenge as a satellite event at MICCAI 2025.<n>We provided participants with a large pretraining dataset, FOMO60K, and evaluated models on data sourced directly from clinical labels.<n>19 foundation models from sixteen teams were evaluated using a standardized containerized pipeline.
  arXiv  Detail & Related papers  (2026-04-13T16:23:46Z)
• Estimating Time Series Foundation Model Transferability via In-Context Learning [74.65355820906355]
  Time series foundation models (TSFMs) offer strong zero-shot forecasting via large-scale pre-training.<n>Fine-tuning remains critical for boosting performance in domains with limited public data.<n>We introduce TimeTic, a transferability estimation framework that recasts model selection as an in-context-learning problem.
  arXiv  Detail & Related papers  (2025-09-28T07:07:13Z)
• Output Scaling: YingLong-Delayed Chain of Thought in a Large Pretrained Time Series Forecasting Model [55.25659103706409]
  This framework achieves state-of-the-art performance for our designed foundation model, YingLong.<n>YingLong is a non-causal, bidirectional attention encoder-only transformer trained through masked token recovery.<n>We release four foundation models ranging from 6M to 300M parameters, demonstrating superior results in zero-shot tasks.
  arXiv  Detail & Related papers  (2025-05-20T14:31:06Z)
• SeisMoLLM: Advancing Seismic Monitoring via Cross-modal Transfer with Pre-trained Large Language Model [69.74609763584449]
  This work presents SeisMoLLM, the first foundation model that utilizes cross-modal transfer for seismic monitoring. It achieves state-of-the-art performance on the DiTing and STEAD datasets across five critical tasks. In addition to its superior performance, SeisMoLLM maintains efficiency comparable to or even better than lightweight models in both training and inference.
  arXiv  Detail & Related papers  (2025-02-27T10:35:53Z)
• Explore the Use of Time Series Foundation Model for Car-Following Behavior Analysis [0.0]
  Modeling car-following behavior is essential for traffic simulation, analyzing driving patterns, and understanding complex traffic flows. While machine learning and deep learning methods capture complex patterns, they require large labeled datasets. Foundation models provide a more efficient alternative, pre-trained on vast, diverse time series datasets.
  arXiv  Detail & Related papers  (2025-01-13T03:13:32Z)
• GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluation [90.53485251837235]
  Time series foundation models excel in zero-shot forecasting, handling diverse tasks without explicit training. GIFT-Eval is a pioneering benchmark aimed at promoting evaluation across diverse datasets. GIFT-Eval encompasses 23 datasets over 144,000 time series and 177 million data points.
  arXiv  Detail & Related papers  (2024-10-14T11:29:38Z)
• Towards a Theoretical Understanding of Memorization in Diffusion Models [76.85077961718875]
  Diffusion probabilistic models (DPMs) are being employed as mainstream models for Generative Artificial Intelligence (GenAI) We provide a theoretical understanding of memorization in both conditional and unconditional DPMs under the assumption of model convergence. We propose a novel data extraction method named textbfSurrogate condItional Data Extraction (SIDE) that leverages a time-dependent classifier trained on the generated data as a surrogate condition to extract training data from unconditional DPMs.
  arXiv  Detail & Related papers  (2024-10-03T13:17:06Z)
• Modeling IoT Traffic Patterns: Insights from a Statistical Analysis of an MTC Dataset [1.2289361708127877]
  Internet-of-Things (IoT) is rapidly expanding, connecting numerous devices and becoming integral to our daily lives. Effective IoT traffic management requires modeling and predicting intrincate machine-type communication (MTC) dynamics. We perform a comprehensive statistical analysis of the MTC traffic utilizing goodness-of-fit tests, including well-established tests such as Kolmogorov-Smirnov, Anderson-Darling, chi-squared, and root mean square error.
  arXiv  Detail & Related papers  (2024-09-03T14:24:18Z)
• Synthesizing Multimodal Electronic Health Records via Predictive Diffusion Models [69.06149482021071]
  We propose a novel EHR data generation model called EHRPD. It is a diffusion-based model designed to predict the next visit based on the current one while also incorporating time interval estimation. We conduct experiments on two public datasets and evaluate EHRPD from fidelity, privacy, and utility perspectives.
  arXiv  Detail & Related papers  (2024-06-20T02:20:23Z)
• Towards a clinically accessible radiology foundation model: open-access and lightweight, with automated evaluation [113.5002649181103]
  Training open-source small multimodal models (SMMs) to bridge competency gaps for unmet clinical needs in radiology. For training, we assemble a large dataset of over 697 thousand radiology image-text pairs. For evaluation, we propose CheXprompt, a GPT-4-based metric for factuality evaluation, and demonstrate its parity with expert evaluation. The inference of LlaVA-Rad is fast and can be performed on a single V100 GPU in private settings, offering a promising state-of-the-art tool for real-world clinical applications.
  arXiv  Detail & Related papers  (2024-03-12T18:12:02Z)
• Task-customized Masked AutoEncoder via Mixture of Cluster-conditional Experts [104.9871176044644]
  Masked Autoencoder(MAE) is a prevailing self-supervised learning method that achieves promising results in model pre-training. We propose a novel MAE-based pre-training paradigm, Mixture of Cluster-conditional Experts (MoCE) MoCE trains each expert only with semantically relevant images by using cluster-conditional gates.
  arXiv  Detail & Related papers  (2024-02-08T03:46:32Z)
• Forecasting Emergency Department Crowding with Advanced Machine Learning Models and Multivariable Input [8.294560133196807]
  Emergency department (ED) crowding is a significant threat to patient safety and it has been repeatedly associated with increased mortality. Despite active research on the subject, several gaps remain: 1) proposed forecasting models have become outdated due to quick influx of advanced machine learning models (ML), 2) amount of multivariable input data has been limited and 3) discrete performance metrics have been rarely reported. We show that N-BEATS and LightGBM outpeform benchmarks with 11 % and 9 % respective improvements and that DeepAR predicts next day crowding with an AUC of 0.76 (95 % CI 0.69-0.84)
  arXiv  Detail & Related papers  (2023-08-31T08:34:20Z)
• EHRSHOT: An EHR Benchmark for Few-Shot Evaluation of Foundation Models [6.506937003687058]
  We publish a new dataset, EHRSHOT, which contains deidentified structured data from the electronic health records (EHRs) of 6,739 patients from Stanford Medicine. Second, we publish the weights of CLMBR-T-base, a 141M parameter clinical foundation model pretrained on the structured EHR data of 2.57M patients. Third, we define 15 few-shot clinical prediction tasks, enabling evaluation of foundation models on benefits such as sample efficiency and task adaptation.
  arXiv  Detail & Related papers  (2023-07-05T05:24:59Z)
• Federated Learning of Medical Concepts Embedding using BEHRT [0.0]
  We propose a federated learning approach for learning medical concepts embedding. Our approach is based on embedding model like BEHRT, a deep neural sequence model for EHR. We compare the performance of a model trained with FL against a model trained on centralized data.
  arXiv  Detail & Related papers  (2023-05-22T14:05:39Z)
• Unsupervised Pre-Training on Patient Population Graphs for Patient-Level Predictions [48.02011627390706]
  Pre-training has shown success in different areas of machine learning, such as Computer Vision (CV), Natural Language Processing (NLP) and medical imaging. In this paper, we apply unsupervised pre-training to heterogeneous, multi-modal EHR data for patient outcome prediction. We find that our proposed graph based pre-training method helps in modeling the data at a population level.
  arXiv  Detail & Related papers  (2022-03-23T17:59:45Z)
• Pre-training transformer-based framework on large-scale pediatric claims data for downstream population-specific tasks [3.1580072841682734]
  This study presents the Claim Pre-Training (Claim-PT) framework, a generic pre-training model that first trains on the entire pediatric claims dataset. The effective knowledge transfer is completed through the task-aware fine-tuning stage. We conducted experiments on a real-world claims dataset with more than one million patient records.
  arXiv  Detail & Related papers  (2021-06-24T15:25:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.