OpenKBP: The open-access knowledge-based planning grand challenge

• URL: http://arxiv.org/abs/2011.14076v2
• Date: Wed, 13 Jan 2021 19:46:18 GMT
• Title: OpenKBP: The open-access knowledge-based planning grand challenge
• Authors: Aaron Babier, Binghao Zhang, Rafid Mahmood, Kevin L. Moore, Thomas G. Purdie, Andrea L. McNiven, Timothy C. Y. Chan
• Abstract summary: We hosted OpenKBP, a 2020 AAPM Grand Challenge, and challenged participants to develop the best method for predicting the dose of contoured CT images. The models were evaluated according to two separate scores: (1) dose score, which evaluates the full 3D dose distributions, and (2) dose-volume histogram (DVH) score, which evaluates a set DVH metrics. The Challenge attracted 195 participants from 28 countries, and 73 of those participants formed 44 teams in the validation phase, which received a total of 1750 submissions.
• Score: 0.6157382820537718
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The purpose of this work is to advance fair and consistent comparisons of dose prediction methods for knowledge-based planning (KBP) in radiation therapy research. We hosted OpenKBP, a 2020 AAPM Grand Challenge, and challenged participants to develop the best method for predicting the dose of contoured CT images. The models were evaluated according to two separate scores: (1) dose score, which evaluates the full 3D dose distributions, and (2) dose-volume histogram (DVH) score, which evaluates a set DVH metrics. Participants were given the data of 340 patients who were treated for head-and-neck cancer with radiation therapy. The data was partitioned into training (n=200), validation (n=40), and testing (n=100) datasets. All participants performed training and validation with the corresponding datasets during the validation phase of the Challenge, and we ranked the models in the testing phase based on out-of-sample performance. The Challenge attracted 195 participants from 28 countries, and 73 of those participants formed 44 teams in the validation phase, which received a total of 1750 submissions. The testing phase garnered submissions from 28 teams. On average, over the course of the validation phase, participants improved the dose and DVH scores of their models by a factor of 2.7 and 5.7, respectively. In the testing phase one model achieved significantly better dose and DVH score than the runner-up models. Lastly, many of the top performing teams reported using generalizable techniques (e.g., ensembles) to achieve higher performance than their competition. This is the first competition for knowledge-based planning research, and it helped launch the first platform for comparing KBP prediction methods fairly and consistently. The OpenKBP datasets are available publicly to help benchmark future KBP research, which has also democratized KBP research by making it accessible to everyone.

Related papers

• SemiHVision: Enhancing Medical Multimodal Models with a Semi-Human Annotated Dataset and Fine-Tuned Instruction Generation [13.672776832197918]
  Multimodal large language models (MLLMs) have made significant strides, yet they face challenges in the medical domain due to limited specialized knowledge. We seek to address this gap at various stages of the end-to-end learning pipeline, including data collection, model fine-tuning, and evaluation.
  arXiv  Detail & Related papers  (2024-10-19T02:35:35Z)
• Towards Efficient Patient Recruitment for Clinical Trials: Application of a Prompt-Based Learning Model [0.7373617024876725]
  Clinical trials are essential for advancing pharmaceutical interventions, but they face a bottleneck in selecting eligible participants. The complex nature of unstructured medical texts presents challenges in efficiently identifying participants. In this study, we aimed to evaluate the performance of a prompt-based large language model for the cohort selection task.
  arXiv  Detail & Related papers  (2024-04-24T20:42:28Z)
• IEEE BigData 2023 Keystroke Verification Challenge (KVC) [14.366081634293721]
  This paper considers the biometric verification performance of Keystroke Dynamics captured as tweet-long sequences of variable transcript text from over 185,000 subjects. The data are obtained from two of the largest public databases of KD up to date. Several neural architectures were proposed by the participants, leading to global Equal Error Rates (EERs) as low as 3.33% and 3.61% achieved by the best team respectively.
  arXiv  Detail & Related papers  (2024-01-29T20:51:42Z)
• The STOIC2021 COVID-19 AI challenge: applying reusable training methodologies to private data [60.94672667514737]
  This study implements the Type Three (T3) challenge format, which allows for training solutions on private data. With T3, challenge organizers train a provided by the participants on sequestered training data. The winning solution obtained an area under the receiver operating characteristic curve for discerning between severe and non-severe COVID-19 of 0.815.
  arXiv  Detail & Related papers  (2023-06-18T05:48:28Z)
• SPOT: Sequential Predictive Modeling of Clinical Trial Outcome with Meta-Learning [67.8195828626489]
  Clinical trials are essential to drug development but time-consuming, costly, and prone to failure. We propose Sequential Predictive mOdeling of clinical Trial outcome (SPOT) that first identifies trial topics to cluster the multi-sourced trial data into relevant trial topics. With the consideration of each trial sequence as a task, it uses a meta-learning strategy to achieve a point where the model can rapidly adapt to new tasks with minimal updates.
  arXiv  Detail & Related papers  (2023-04-07T23:04:27Z)
• Biomedical image analysis competitions: The state of current participation practice [143.52578599912326]
  We designed a survey to shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis. The survey covered participants' expertise and working environments, their chosen strategies, as well as algorithm characteristics. Overall, 94% of all solutions were deep learning-based. Of these, 84% were based on standard architectures.
  arXiv  Detail & Related papers  (2022-12-16T16:44:46Z)
• Lung-Originated Tumor Segmentation from Computed Tomography Scan (LOTUS) Benchmark [48.30502612686276]
  Lung cancer is one of the deadliest cancers, and its effective diagnosis and treatment depend on the accurate delineation of the tumor. Human-centered segmentation, which is currently the most common approach, is subject to inter-observer variability. The 2018 VIP Cup started with a global engagement from 42 countries to access the competition data. In a nutshell, all the algorithms proposed during the competition, are based on deep learning models combined with a false positive reduction technique.
  arXiv  Detail & Related papers  (2022-01-03T03:06:38Z)
• Analysing Affective Behavior in the second ABAW2 Competition [70.86998050535944]
  The Affective Behavior Analysis in-the-wild (ABAW2) 2021 Competition is the second -- following the first very successful ABAW Competition held in conjunction with IEEE FG 2020- Competition that aims at automatically analyzing affect.
  arXiv  Detail & Related papers  (2021-06-14T11:30:19Z)
• FairFace Challenge at ECCV 2020: Analyzing Bias in Face Recognition [26.49981022316179]
  The aim of the challenge was to evaluate accuracy and bias in gender and skin colour of submitted algorithms. The dataset is not balanced, which simulates a real world scenario where AI-based models supposed to present fair outcomes are trained and evaluated on imbalanced data. The analysis of top-10 teams shows higher false positive rates (and lower false negative rates) for females with dark skin tone.
  arXiv  Detail & Related papers  (2020-09-16T17:56:22Z)
• Analysing Affective Behavior in the First ABAW 2020 Competition [49.90617840789334]
  The Affective Behavior Analysis in-the-wild (ABAW) 2020 Competition is the first Competition aiming at automatic analysis of the three main behavior tasks. We describe this Competition, to be held in conjunction with the IEEE Conference on Face and Gesture Recognition, May 2020, in Buenos Aires, Argentina. We outline the evaluation metrics, present both the baseline system and the top-3 performing teams' methodologies per Challenge and finally present their obtained results.
  arXiv  Detail & Related papers  (2020-01-30T15:41:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.