Arges: Spatio-Temporal Transformer for Ulcerative Colitis Severity Assessment in Endoscopy Videos

• URL: http://arxiv.org/abs/2410.00536v1
• Date: Tue, 1 Oct 2024 09:23:14 GMT
• Title: Arges: Spatio-Temporal Transformer for Ulcerative Colitis Severity Assessment in Endoscopy Videos
• Authors: Krishna Chaitanya, Pablo F. Damasceno, Shreyas Fadnavis, Pooya Mobadersany, Chaitanya Parmar, Emily Scherer, Natalia Zemlianskaia, Lindsey Surace, Louis R. Ghanem, Oana Gabriela Cula, Tommaso Mansi, Kristopher Standish,
• Abstract summary: Expert MES/UCEIS annotation is time-consuming and susceptible to inter-rater variability. CNN-based weakly-supervised models with end-to-end (e2e) training lack generalization to new disease scores. "Arges" is a deep learning framework that incorporates positional encoding to estimate disease severity scores in endoscopy.
• Score: 2.0735422289416605
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accurate assessment of disease severity from endoscopy videos in ulcerative colitis (UC) is crucial for evaluating drug efficacy in clinical trials. Severity is often measured by the Mayo Endoscopic Subscore (MES) and Ulcerative Colitis Endoscopic Index of Severity (UCEIS) score. However, expert MES/UCEIS annotation is time-consuming and susceptible to inter-rater variability, factors addressable by automation. Automation attempts with frame-level labels face challenges in fully-supervised solutions due to the prevalence of video-level labels in clinical trials. CNN-based weakly-supervised models (WSL) with end-to-end (e2e) training lack generalization to new disease scores and ignore spatio-temporal information crucial for accurate scoring. To address these limitations, we propose "Arges", a deep learning framework that utilizes a transformer with positional encoding to incorporate spatio-temporal information from frame features to estimate disease severity scores in endoscopy video. Extracted features are derived from a foundation model (ArgesFM), pre-trained on a large diverse dataset from multiple clinical trials (61M frames, 3927 videos). We evaluate four UC disease severity scores, including MES and three UCEIS component scores. Test set evaluation indicates significant improvements, with F1 scores increasing by 4.1% for MES and 18.8%, 6.6%, 3.8% for the three UCEIS component scores compared to state-of-the-art methods. Prospective validation on previously unseen clinical trial data further demonstrates the model's successful generalization.

Related papers

• FedMedICL: Towards Holistic Evaluation of Distribution Shifts in Federated Medical Imaging [68.6715007665896]
  FedMedICL is a unified framework and benchmark to holistically evaluate federated medical imaging challenges. We comprehensively evaluate several popular methods on six diverse medical imaging datasets. We find that a simple batch balancing technique surpasses advanced methods in average performance across FedMedICL experiments.
  arXiv  Detail & Related papers  (2024-07-11T19:12:23Z)
• Fusion of Diffusion Weighted MRI and Clinical Data for Predicting Functional Outcome after Acute Ischemic Stroke with Deep Contrastive Learning [1.4149937986822438]
  Stroke is a common disabling neurological condition that affects about one-quarter of the adult population over age 25. Our proposed fusion model achieves 0.87, 0.80 and 80.45% for AUC, F1-score and accuracy, respectively.
  arXiv  Detail & Related papers  (2024-02-16T18:51:42Z)
• Improving Multiple Sclerosis Lesion Segmentation Across Clinical Sites: A Federated Learning Approach with Noise-Resilient Training [75.40980802817349]
  Deep learning models have shown promise for automatically segmenting MS lesions, but the scarcity of accurately annotated data hinders progress in this area. We introduce a Decoupled Hard Label Correction (DHLC) strategy that considers the imbalanced distribution and fuzzy boundaries of MS lesions. We also introduce a Centrally Enhanced Label Correction (CELC) strategy, which leverages the aggregated central model as a correction teacher for all sites.
  arXiv  Detail & Related papers  (2023-08-31T00:36:10Z)
• SPOT: Sequential Predictive Modeling of Clinical Trial Outcome with Meta-Learning [67.8195828626489]
  Clinical trials are essential to drug development but time-consuming, costly, and prone to failure. We propose Sequential Predictive mOdeling of clinical Trial outcome (SPOT) that first identifies trial topics to cluster the multi-sourced trial data into relevant trial topics. With the consideration of each trial sequence as a task, it uses a meta-learning strategy to achieve a point where the model can rapidly adapt to new tasks with minimal updates.
  arXiv  Detail & Related papers  (2023-04-07T23:04:27Z)
• Patch-level instance-group discrimination with pretext-invariant learning for colitis scoring [2.691339855008848]
  We introduce a novel patch-level instance-group discrimination with pretext-invariant representation learning (PLD-PIRL) for self-supervised learning (SSL) Our experiments demonstrate both improved accuracy and robustness compared to the baseline supervised network and several state-of-the-art SSL methods.
  arXiv  Detail & Related papers  (2022-07-11T21:06:29Z)
• Visual Acuity Prediction on Real-Life Patient Data Using a Machine Learning Based Multistage System [0.40151799356083057]
  The prediction of the visual acuity (VA) and the earliest possible detection of deterioration under real-life conditions is challenging due to heterogeneous and incomplete data. We present a workflow for the development of a research-compatible data corpus fusing different IT systems of the department of ophthalmology of a German maximum care hospital. We achieve a final prediction accuracy of 69 % in macro average F1-score, while being in the same range as the ophthalmologists with 57.8 and 50 +- 10.7 % F1-score.
  arXiv  Detail & Related papers  (2022-04-25T21:20:27Z)
• Automatic Estimation of Ulcerative Colitis Severity from Endoscopy Videos using Ordinal Multi-Instance Learning [50.591267188664666]
  Ulcerative colitis (UC) is a chronic inflammatory bowel disease characterized by relapsing inflammation of the large intestine. The severity of UC is often represented by the Mayo Endoscopic Subscore (MES) which quantifies mucosal disease activity from endoscopy videos. We propose a novel weakly supervised, ordinal classification method to estimate frame severity from video MES labels alone.
  arXiv  Detail & Related papers  (2021-09-29T19:42:51Z)
• Cross-Site Severity Assessment of COVID-19 from CT Images via Domain Adaptation [64.59521853145368]
  Early and accurate severity assessment of Coronavirus disease 2019 (COVID-19) based on computed tomography (CT) images offers a great help to the estimation of intensive care unit event. To augment the labeled data and improve the generalization ability of the classification model, it is necessary to aggregate data from multiple sites. This task faces several challenges including class imbalance between mild and severe infections, domain distribution discrepancy between sites, and presence of heterogeneous features.
  arXiv  Detail & Related papers  (2021-09-08T07:56:51Z)
• Explainability Guided Multi-Site COVID-19 CT Classification [79.4957965474334]
  The limited number of supervised positive cases, the lack of region-based supervision, and the variability across acquisition sites are addressed. Compared to the current state of the art, we obtain an increase of five percent in the F1 score on a site with a relatively high number of cases, and a gap twice as large for a site with much fewer training images.
  arXiv  Detail & Related papers  (2021-03-25T08:56:08Z)
• Validation and Optimization of Multi-Organ Segmentation on Clinical Imaging Archives [7.036733782879497]
  A 2015 MICCAI challenge spurred substantial innovation in multi-organ abdominal CT segmentation. Recent innovations in deep methods have driven performance toward levels for which clinical translation is appealing. Cross-validation on open datasets presents the risk of indirect knowledge contamination and could result in circular reasoning.
  arXiv  Detail & Related papers  (2020-02-10T21:49:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.