One-shot skill assessment in high-stakes domains with limited data via meta learning

• URL: http://arxiv.org/abs/2301.00812v5
• Date: Fri, 19 Apr 2024 16:10:40 GMT
• Title: One-shot skill assessment in high-stakes domains with limited data via meta learning
• Authors: Erim Yanik, Steven Schwaitzberg, Gene Yang, Xavier Intes, Jack Norfleet, Matthew Hackett, Suvranu De,
• Abstract summary: A-VBANet is a novel meta-learning model capable of delivering domain-agnostic skill assessment via one-shot learning. Our model successfully adapted with accuracies up to 99.5% in one-shot and 99.9% in few-shot settings for simulated tasks and 89.7% for laparoscopic cholecystectomy.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep Learning (DL) has achieved robust competency assessment in various high-stakes fields. However, the applicability of DL models is often hampered by their substantial data requirements and confinement to specific training domains. This prevents them from transitioning to new tasks where data is scarce. Therefore, domain adaptation emerges as a critical element for the practical implementation of DL in real-world scenarios. Herein, we introduce A-VBANet, a novel meta-learning model capable of delivering domain-agnostic skill assessment via one-shot learning. Our methodology has been tested by assessing surgical skills on five laparoscopic and robotic simulators and real-life laparoscopic cholecystectomy. Our model successfully adapted with accuracies up to 99.5% in one-shot and 99.9% in few-shot settings for simulated tasks and 89.7% for laparoscopic cholecystectomy. This study marks the first instance of a domain-agnostic methodology for skill assessment in critical fields setting a precedent for the broad application of DL across diverse real-life domains with limited data.

Related papers

• Exploring Pre-training Across Domains for Few-Shot Surgical Skill Assessment [6.796181867667279]
  Automated surgical skill assessment (SSA) is a central task in surgical computer vision.<n>Few-shot learning offers a scalable alternative enabling model development with minimal supervision.<n>While widely studied for several surgical downstream tasks, pre-training has remained largely unexplored in SSA.
  arXiv  Detail & Related papers  (2025-09-11T10:23:19Z)
• Evaluating Self-Supervised Learning in Medical Imaging: A Benchmark for Robustness, Generalizability, and Multi-Domain Impact [0.3141085922386211]
  We present a comprehensive evaluation of self-supervised learning (SSL) methods within the medical domain. Using the MedMNIST dataset collection as a standardized benchmark, we evaluate 8 major SSL methods across 11 different medical datasets.
  arXiv  Detail & Related papers  (2024-12-26T08:51:56Z)
• Weakly supervised deep learning model with size constraint for prostate cancer detection in multiparametric MRI and generalization to unseen domains [0.90668179713299]
  We show that the model achieves on-par performance with strong fully supervised baseline models. We also observe a performance decrease for both fully supervised and weakly supervised models when tested on unseen data domains.
  arXiv  Detail & Related papers  (2024-11-04T12:24:33Z)
• Day-Night Adaptation: An Innovative Source-free Adaptation Framework for Medical Image Segmentation [51.520294290813865]
  We propose a novel adaptation framework called Day-Night Adaptation (DyNA) with insights. During the day, a low-frequency prompt is trained to adapt the frozen model to each test sample. During the night, we reuse test data collected from the day and introduce a global student model to bridge the knowledge between teacher and student models.
  arXiv  Detail & Related papers  (2024-10-17T12:02:29Z)
• Domain Adaptation of Echocardiography Segmentation Via Reinforcement Learning [4.850478245721347]
  We introduce RL4Seg, an innovative reinforcement learning framework that reduces the need to otherwise incorporate large expertly annotated datasets in the target domain. Using a target dataset of 10,000 unannotated 2D echocardiographic images, RL4Seg achieves 99% anatomical validity on a subset of 220 expert-validated subjects from the target domain.
  arXiv  Detail & Related papers  (2024-06-25T19:26:39Z)
• ProtoAL: Interpretable Deep Active Learning with prototypes for medical imaging [0.6292138336765966]
  We propose the ProtoAL method, where we integrate an interpretable DL model into the Deep Active Learning framework. We evaluated ProtoAL on the Messidor dataset, achieving an area under the precision-recall curve of 0.79 while utilizing only 76.54% of the available labeled data.
  arXiv  Detail & Related papers  (2024-04-06T21:39:49Z)
• Towards a clinically accessible radiology foundation model: open-access and lightweight, with automated evaluation [113.5002649181103]
  Training open-source small multimodal models (SMMs) to bridge competency gaps for unmet clinical needs in radiology. For training, we assemble a large dataset of over 697 thousand radiology image-text pairs. For evaluation, we propose CheXprompt, a GPT-4-based metric for factuality evaluation, and demonstrate its parity with expert evaluation. The inference of LlaVA-Rad is fast and can be performed on a single V100 GPU in private settings, offering a promising state-of-the-art tool for real-world clinical applications.
  arXiv  Detail & Related papers  (2024-03-12T18:12:02Z)
• Jumpstarting Surgical Computer Vision [2.585559512929966]
  We develop recommendations for pretraining dataset composition through over 300 experiments.<n>We outperform state-of-the-art pre-trainings on two public benchmarks for phase recognition.
  arXiv  Detail & Related papers  (2023-12-10T18:54:16Z)
• One-Shot Domain Adaptive and Generalizable Semantic Segmentation with Class-Aware Cross-Domain Transformers [96.51828911883456]
  Unsupervised sim-to-real domain adaptation (UDA) for semantic segmentation aims to improve the real-world test performance of a model trained on simulated data. Traditional UDA often assumes that there are abundant unlabeled real-world data samples available during training for the adaptation. We explore the one-shot unsupervised sim-to-real domain adaptation (OSUDA) and generalization problem, where only one real-world data sample is available.
  arXiv  Detail & Related papers  (2022-12-14T15:54:15Z)
• Dissecting Self-Supervised Learning Methods for Surgical Computer Vision [51.370873913181605]
  Self-Supervised Learning (SSL) methods have begun to gain traction in the general computer vision community. The effectiveness of SSL methods in more complex and impactful domains, such as medicine and surgery, remains limited and unexplored. We present an extensive analysis of the performance of these methods on the Cholec80 dataset for two fundamental and popular tasks in surgical context understanding, phase recognition and tool presence detection.
  arXiv  Detail & Related papers  (2022-07-01T14:17:11Z)
• Ensembling Handcrafted Features with Deep Features: An Analytical Study for Classification of Routine Colon Cancer Histopathological Nuclei Images [13.858624044986815]
  We have used F1-measure, Precision, Recall, AUC, and Cross-Entropy Loss to analyse the performance of our approaches. We observed from the results that the DL features ensemble bring a marked improvement in the overall performance of the model.
  arXiv  Detail & Related papers  (2022-02-22T06:48:50Z)
• Enhancing Clinical Information Extraction with Transferred Contextual Embeddings [9.143551270841858]
  Bidirectional Representations from Transformers (BERT) model has achieved the state-of-the-art performance for many natural language processing (NLP) tasks. We show that BERT based pre-training models can be transferred to health-related documents under mild conditions.
  arXiv  Detail & Related papers  (2021-09-15T12:22:57Z)
• Domain Generalization on Medical Imaging Classification using Episodic Training with Task Augmentation [62.49837463676111]
  We propose a novel scheme of episodic training with task augmentation on medical imaging classification. Motivated by the limited number of source domains in real-world medical deployment, we consider the unique task-level overfitting.
  arXiv  Detail & Related papers  (2021-06-13T03:56:59Z)
• Shape-aware Meta-learning for Generalizing Prostate MRI Segmentation to Unseen Domains [68.73614619875814]
  We present a novel shape-aware meta-learning scheme to improve the model generalization in prostate MRI segmentation. Experimental results show that our approach outperforms many state-of-the-art generalization methods consistently across all six settings of unseen domains.
  arXiv  Detail & Related papers  (2020-07-04T07:56:02Z)
• Self-Training with Improved Regularization for Sample-Efficient Chest X-Ray Classification [80.00316465793702]
  We present a deep learning framework that enables robust modeling in challenging scenarios. Our results show that using 85% lesser labeled data, we can build predictive models that match the performance of classifiers trained in a large-scale data setting.
  arXiv  Detail & Related papers  (2020-05-03T02:36:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.