Exploring a Graph-based Approach to Offline Reinforcement Learning for Sepsis Treatment

• URL: http://arxiv.org/abs/2509.03393v1
• Date: Wed, 03 Sep 2025 15:15:41 GMT
• Title: Exploring a Graph-based Approach to Offline Reinforcement Learning for Sepsis Treatment
• Authors: Taisiya Khakharova, Lucas Sakizloglou, Leen Lambers,
• Abstract summary: This study models patient data from the well-known MIMIC-III dataset as a heterogeneous graph that evolves over time.<n>We explore two Graph Neural Network architectures - GraphSAGE and GATv2 - for learning patient state representations.
• Score: 2.7655615458735174
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Sepsis is a serious, life-threatening condition. When treating sepsis, it is challenging to determine the correct amount of intravenous fluids and vasopressors for a given patient. While automated reinforcement learning (RL)-based methods have been used to support these decisions with promising results, previous studies have relied on relational data. Given the complexity of modern healthcare data, representing data as a graph may provide a more natural and effective approach. This study models patient data from the well-known MIMIC-III dataset as a heterogeneous graph that evolves over time. Subsequently, we explore two Graph Neural Network architectures - GraphSAGE and GATv2 - for learning patient state representations, adopting the approach of decoupling representation learning from policy learning. The encoders are trained to produce latent state representations, jointly with decoders that predict the next patient state. These representations are then used for policy learning with the dBCQ algorithm. The results of our experimental evaluation confirm the potential of a graph-based approach, while highlighting the complexity of representation learning in this domain.

Related papers

• AdaMedGraph: Adaboosting Graph Neural Networks for Personalized Medicine [31.424781716926848]
  We propose a novel algorithm named ours, which can automatically select important features to construct multiple patient similarity graphs. ours is evaluated on two real-world medical scenarios and shows superiors performance.
  arXiv  Detail & Related papers  (2023-11-24T06:27:25Z)
• Graph AI in Medicine [9.733108180046555]
  Graph neural networks (GNNs) process data holistically by viewing modalities as nodes interconnected by their relationships. GNNs capture information through localized neural transformations defined on graph relationships. Knowledge graphs can enhance interpretability by aligning model-driven insights with medical knowledge.
  arXiv  Detail & Related papers  (2023-10-20T19:01:01Z)
• Graph data modelling for outcome prediction in oropharyngeal cancer patients [38.37247384790338]
  Graph neural networks (GNNs) are becoming increasingly popular in the medical domain for the tasks of disease classification and outcome prediction. We propose a patient hypergraph network (PHGN) which has been investigated in an inductive learning setup for binary outcome prediction in oropharyngeal cancer (OPC) patients.
  arXiv  Detail & Related papers  (2023-10-04T16:09:35Z)
• Graph-in-Graph (GiG): Learning interpretable latent graphs in non-Euclidean domain for biological and healthcare applications [52.65389473899139]
  Graphs are a powerful tool for representing and analyzing unstructured, non-Euclidean data ubiquitous in the healthcare domain. Recent works have shown that considering relationships between input data samples have a positive regularizing effect for the downstream task. We propose Graph-in-Graph (GiG), a neural network architecture for protein classification and brain imaging applications.
  arXiv  Detail & Related papers  (2022-04-01T10:01:37Z)
• MIMO: Mutual Integration of Patient Journey and Medical Ontology for Healthcare Representation Learning [49.57261599776167]
  We propose an end-to-end robust Transformer-based solution, Mutual Integration of patient journey and Medical Ontology (MIMO) for healthcare representation learning and predictive analytics.
  arXiv  Detail & Related papers  (2021-07-20T07:04:52Z)
• Multi-modal Graph Learning for Disease Prediction [35.4310911850558]
  We propose an end-to-end Multimodal Graph Learning framework (MMGL) for disease prediction. Instead of defining the adjacency matrix manually as existing methods, the latent graph structure can be captured through a novel way of adaptive graph learning.
  arXiv  Detail & Related papers  (2021-07-01T03:59:22Z)
• Deep Co-Attention Network for Multi-View Subspace Learning [73.3450258002607]
  We propose a deep co-attention network for multi-view subspace learning. It aims to extract both the common information and the complementary information in an adversarial setting. In particular, it uses a novel cross reconstruction loss and leverages the label information to guide the construction of the latent representation.
  arXiv  Detail & Related papers  (2021-02-15T18:46:44Z)
• An Empirical Study of Representation Learning for Reinforcement Learning in Healthcare [19.50370829781689]
  We use data from septic patients in the MIMIC-III dataset to form representations of a patient state. We find that sequentially formed state representations facilitate effective policy learning in batch settings.
  arXiv  Detail & Related papers  (2020-11-23T06:37:08Z)
• BiteNet: Bidirectional Temporal Encoder Network to Predict Medical Outcomes [53.163089893876645]
  We propose a novel self-attention mechanism that captures the contextual dependency and temporal relationships within a patient's healthcare journey. An end-to-end bidirectional temporal encoder network (BiteNet) then learns representations of the patient's journeys. We have evaluated the effectiveness of our methods on two supervised prediction and two unsupervised clustering tasks with a real-world EHR dataset.
  arXiv  Detail & Related papers  (2020-09-24T00:42:36Z)
• Uncovering the structure of clinical EEG signals with self-supervised learning [64.4754948595556]
  Supervised learning paradigms are often limited by the amount of labeled data that is available. This phenomenon is particularly problematic in clinically-relevant data, such as electroencephalography (EEG) By extracting information from unlabeled data, it might be possible to reach competitive performance with deep neural networks.
  arXiv  Detail & Related papers  (2020-07-31T14:34:47Z)
• Learning Dynamic and Personalized Comorbidity Networks from Event Data using Deep Diffusion Processes [102.02672176520382]
  Comorbid diseases co-occur and progress via complex temporal patterns that vary among individuals. In electronic health records we can observe the different diseases a patient has, but can only infer the temporal relationship between each co-morbid condition. We develop deep diffusion processes to model "dynamic comorbidity networks"
  arXiv  Detail & Related papers  (2020-01-08T15:47:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.