The Concordance Index decomposition: A measure for a deeper understanding of survival prediction models

• URL: http://arxiv.org/abs/2203.00144v3
• Date: Sat, 20 Jan 2024 21:46:23 GMT
• Title: The Concordance Index decomposition: A measure for a deeper understanding of survival prediction models
• Authors: Abdallah Alabdallah, Mattias Ohlsson, Sepideh Pashami, Thorsteinn R\"ognvaldsson
• Abstract summary: The Concordance Index (C-index) is a commonly used metric in Survival Analysis for evaluating the performance of a prediction model. We propose a decomposition of the C-index into a weighted harmonic mean of two quantities: one for ranking observed events versus other observed events, and the other for ranking observed events versus censored cases.
• Score: 3.186455928607442
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The Concordance Index (C-index) is a commonly used metric in Survival Analysis for evaluating the performance of a prediction model. In this paper, we propose a decomposition of the C-index into a weighted harmonic mean of two quantities: one for ranking observed events versus other observed events, and the other for ranking observed events versus censored cases. This decomposition enables a finer-grained analysis of the relative strengths and weaknesses between different survival prediction methods. The usefulness of this decomposition is demonstrated through benchmark comparisons against classical models and state-of-the-art methods, together with the new variational generative neural-network-based method (SurVED) proposed in this paper. The performance of the models is assessed using four publicly available datasets with varying levels of censoring. Using the C-index decomposition and synthetic censoring, the analysis shows that deep learning models utilize the observed events more effectively than other models. This allows them to keep a stable C-index in different censoring levels. In contrast to such deep learning methods, classical machine learning models deteriorate when the censoring level decreases due to their inability to improve on ranking the events versus other events.

Related papers

• Distilling Influences to Mitigate Prediction Churn in Graph Neural Networks [4.213427823201119]
  Models with similar performances exhibit significant disagreement in the predictions of individual samples, referred to as prediction churn. We propose a novel metric called Influence Difference (ID) to quantify the variation in reasons used by nodes across models. We also consider the differences between nodes with a stable and an unstable prediction, positing that both equally utilize different reasons. As an efficient approximation, we introduce DropDistillation (DD) that matches the output for a graph perturbed by edge deletions.
  arXiv  Detail & Related papers  (2023-10-02T07:37:28Z)
• Predictive change point detection for heterogeneous data [1.1720726814454114]
  "Predict and Compare" is a change point detection framework assisted by a predictive machine learning model. It outperforms online CPD routines in terms of false positive rate and out-of-control average run length. The power of the method is demonstrated in a tribological case study.
  arXiv  Detail & Related papers  (2023-05-11T07:59:18Z)
• A Statistical Learning Take on the Concordance Index for Survival Analysis [0.29005223064604074]
  We provide C-index Fisher-consistency results and excess risk bounds for several commonly used cost functions in survival analysis. We also study the general case where no model assumption is made and present a new, off-the-shelf method that is shown to be consistent with the C-index.
  arXiv  Detail & Related papers  (2023-02-23T14:33:54Z)
• Testing the Robustness of Learned Index Structures [15.472214703318805]
  This work evaluates the robustness of learned index structures in the presence of adversarial workloads. To simulate adversarial workloads, we carry out a data poisoning attack on linear regression models. We show that learned index structures can suffer from a significant performance deterioration of up to 20% when evaluated on poisoned vs. non-poisoned datasets.
  arXiv  Detail & Related papers  (2022-07-23T18:44:54Z)
• Compound virtual screening by learning-to-rank with gradient boosting decision tree and enrichment-based cumulative gain [0.0]
  gradient boosting decision tree (GBDT)-based learning-to-rank methods have gained popularity recently. Normalized Enrichment Discounted Cumulative Gain (NEDCG) aims to properly evaluate the goodness of ranking predictions. NEDCG showed that predictions by regression were comparable to random predictions in multi-assay, multi-family datasets.
  arXiv  Detail & Related papers  (2022-05-04T16:36:24Z)
• Revisiting Consistency Regularization for Semi-Supervised Learning [80.28461584135967]
  We propose an improved consistency regularization framework by a simple yet effective technique, FeatDistLoss. Experimental results show that our model defines a new state of the art for various datasets and settings.
  arXiv  Detail & Related papers  (2021-12-10T20:46:13Z)
• Instance-Level Relative Saliency Ranking with Graph Reasoning [126.09138829920627]
  We present a novel unified model to segment salient instances and infer relative saliency rank order. A novel loss function is also proposed to effectively train the saliency ranking branch. experimental results demonstrate that our proposed model is more effective than previous methods.
  arXiv  Detail & Related papers  (2021-07-08T13:10:42Z)
• A comprehensive comparative evaluation and analysis of Distributional Semantic Models [61.41800660636555]
  We perform a comprehensive evaluation of type distributional vectors, either produced by static DSMs or obtained by averaging the contextualized vectors generated by BERT. The results show that the alleged superiority of predict based models is more apparent than real, and surely not ubiquitous. We borrow from cognitive neuroscience the methodology of Representational Similarity Analysis (RSA) to inspect the semantic spaces generated by distributional models.
  arXiv  Detail & Related papers  (2021-05-20T15:18:06Z)
• Firearm Detection via Convolutional Neural Networks: Comparing a Semantic Segmentation Model Against End-to-End Solutions [68.8204255655161]
  Threat detection of weapons and aggressive behavior from live video can be used for rapid detection and prevention of potentially deadly incidents. One way for achieving this is through the use of artificial intelligence and, in particular, machine learning for image analysis. We compare a traditional monolithic end-to-end deep learning model and a previously proposed model based on an ensemble of simpler neural networks detecting fire-weapons via semantic segmentation.
  arXiv  Detail & Related papers  (2020-12-17T15:19:29Z)
• Explaining and Improving Model Behavior with k Nearest Neighbor Representations [107.24850861390196]
  We propose using k nearest neighbor representations to identify training examples responsible for a model's predictions. We show that kNN representations are effective at uncovering learned spurious associations. Our results indicate that the kNN approach makes the finetuned model more robust to adversarial inputs.
  arXiv  Detail & Related papers  (2020-10-18T16:55:25Z)
• Learning from Aggregate Observations [82.44304647051243]
  We study the problem of learning from aggregate observations where supervision signals are given to sets of instances. We present a general probabilistic framework that accommodates a variety of aggregate observations. Simple maximum likelihood solutions can be applied to various differentiable models.
  arXiv  Detail & Related papers  (2020-04-14T06:18:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.