Active Prompt Tuning Enables Gpt-40 To Do Efficient Classification Of Microscopy Images

• URL: http://arxiv.org/abs/2411.02639v1
• Date: Mon, 04 Nov 2024 21:56:48 GMT
• Title: Active Prompt Tuning Enables Gpt-40 To Do Efficient Classification Of Microscopy Images
• Authors: Abhiram Kandiyana, Peter R. Mouton, Yaroslav Kolinko, Lawrence O. Hall, Dmitry Goldgof,
• Abstract summary: Traditional deep learning-based methods for classifying cellular features in microscopy images require time- and labor-intensive processes for training models. We previously proposed a solution that overcomes these challenges using OpenAI's GPT-4(V) model on a pilot dataset. Results on the pilot dataset were equivalent in accuracy and with a substantial improvement in throughput efficiency compared to the baseline.
• Score: 0.0
• License:
• Abstract: Traditional deep learning-based methods for classifying cellular features in microscopy images require time- and labor-intensive processes for training models. Among the current limitations are major time commitments from domain experts for accurate ground truth preparation; and the need for a large amount of input image data. We previously proposed a solution that overcomes these challenges using OpenAI's GPT-4(V) model on a pilot dataset (Iba-1 immuno-stained tissue sections from 11 mouse brains). Results on the pilot dataset were equivalent in accuracy and with a substantial improvement in throughput efficiency compared to the baseline using a traditional Convolutional Neural Net (CNN)-based approach. The present study builds upon this framework using a second unique and substantially larger dataset of microscopy images. Our current approach uses a newer and faster model, GPT-4o, along with improved prompts. It was evaluated on a microscopy image dataset captured at low (10x) magnification from cresyl-violet-stained sections through the cerebellum of a total of 18 mouse brains (9 Lurcher mice, 9 wild-type controls). We used our approach to classify these images either as a control group or Lurcher mutant. Using 6 mice in the prompt set the results were correct classification for 11 out of the 12 mice (92%) with 96% higher efficiency, reduced image requirements, and lower demands on time and effort of domain experts compared to the baseline method (snapshot ensemble of CNN models). These results confirm that our approach is effective across multiple datasets from different brain regions and magnifications, with minimal overhead.

Related papers

• Masked Autoencoders for Microscopy are Scalable Learners of Cellular Biology [2.7280901660033643]
  This work explores the scaling properties of weakly supervised classifiers and self-supervised masked autoencoders (MAEs) Our results show that ViT-based MAEs outperform weakly supervised classifiers on a variety of tasks, achieving as much as a 11.5% relative improvement when recalling known biological relationships curated from public databases. We develop a new channel-agnostic MAE architecture (CA-MAE) that allows for inputting images of different numbers and orders of channels at inference time.
  arXiv  Detail & Related papers  (2024-04-16T02:42:06Z)
• Affine-Consistent Transformer for Multi-Class Cell Nuclei Detection [76.11864242047074]
  We propose a novel Affine-Consistent Transformer (AC-Former), which directly yields a sequence of nucleus positions. We introduce an Adaptive Affine Transformer (AAT) module, which can automatically learn the key spatial transformations to warp original images for local network training. Experimental results demonstrate that the proposed method significantly outperforms existing state-of-the-art algorithms on various benchmarks.
  arXiv  Detail & Related papers  (2023-10-22T02:27:02Z)
• No Data Augmentation? Alternative Regularizations for Effective Training on Small Datasets [0.0]
  We study alternative regularization strategies to push the limits of supervised learning on small image classification datasets. In particular, we employ a agnostic to select (semi) optimal learning rate and weight decay couples via the norm of model parameters. We reach a test accuracy of 66.5%, on par with the best state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-09-04T16:13:59Z)
• One-Shot Learning for Periocular Recognition: Exploring the Effect of Domain Adaptation and Data Bias on Deep Representations [59.17685450892182]
  We investigate the behavior of deep representations in widely used CNN models under extreme data scarcity for One-Shot periocular recognition. We improved state-of-the-art results that made use of networks trained with biometric datasets with millions of images. Traditional algorithms like SIFT can outperform CNNs in situations with limited data.
  arXiv  Detail & Related papers  (2023-07-11T09:10:16Z)
• Optimizations of Autoencoders for Analysis and Classification of Microscopic In Situ Hybridization Images [68.8204255655161]
  We propose a deep-learning framework to detect and classify areas of microscopic images with similar levels of gene expression. The data we analyze requires an unsupervised learning model for which we employ a type of Artificial Neural Network - Deep Learning Autoencoders.
  arXiv  Detail & Related papers  (2023-04-19T13:45:28Z)
• Performance of GAN-based augmentation for deep learning COVID-19 image classification [57.1795052451257]
  The biggest challenge in the application of deep learning to the medical domain is the availability of training data. Data augmentation is a typical methodology used in machine learning when confronted with a limited data set. In this work, a StyleGAN2-ADA model of Generative Adversarial Networks is trained on the limited COVID-19 chest X-ray image set.
  arXiv  Detail & Related papers  (2023-04-18T15:39:58Z)
• Localization and Classification of Parasitic Eggs in Microscopic Images Using an EfficientDet Detector [0.0]
  We propose a multi-modal learning detector to localize parasitic eggs and categorize them into 11 categories. Our results show robust performance with an accuracy of 92%, and an F1 score of 93%.
  arXiv  Detail & Related papers  (2022-08-03T10:28:18Z)
• Core Risk Minimization using Salient ImageNet [53.616101711801484]
  We introduce the Salient Imagenet dataset with more than 1 million soft masks localizing core and spurious features for all 1000 Imagenet classes. Using this dataset, we first evaluate the reliance of several Imagenet pretrained models (42 total) on spurious features. Next, we introduce a new learning paradigm called Core Risk Minimization (CoRM) whose objective ensures that the model predicts a class using its core features.
  arXiv  Detail & Related papers  (2022-03-28T01:53:34Z)
• Classification of Microscopy Images of Breast Tissue: Region Duplication based Self-Supervision vs. Off-the Shelf Deep Representations [0.0]
  We propose a novel self-supervision pretext task to train a convolutional neural network (CNN) and extract domain specific features. Results indicated that the best performance of 99% sensitivity was achieved for the deep features extracted using ResNet50 with concatenation of patch-level embedding.
  arXiv  Detail & Related papers  (2022-02-12T14:12:13Z)
• Classification of COVID-19 in CT Scans using Multi-Source Transfer Learning [91.3755431537592]
  We propose the use of Multi-Source Transfer Learning to improve upon traditional Transfer Learning for the classification of COVID-19 from CT scans. With our multi-source fine-tuning approach, our models outperformed baseline models fine-tuned with ImageNet. Our best performing model was able to achieve an accuracy of 0.893 and a Recall score of 0.897, outperforming its baseline Recall score by 9.3%.
  arXiv  Detail & Related papers  (2020-09-22T11:53:06Z)
• Learning to segment clustered amoeboid cells from brightfield microscopy via multi-task learning with adaptive weight selection [6.836162272841265]
  We introduce a novel supervised technique for cell segmentation in a multi-task learning paradigm. A combination of a multi-task loss, based on the region and cell boundary detection, is employed for an improved prediction efficiency of the network. We observe an overall Dice score of 0.93 on the validation set, which is an improvement of over 15.9% on a recent unsupervised method, and outperforms the popular supervised U-net algorithm by at least $5.8%$ on average.
  arXiv  Detail & Related papers  (2020-05-19T11:31:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.