Adapting Segment Anything Model (SAM) through Prompt-based Learning for Enhanced Protein Identification in Cryo-EM Micrographs

• URL: http://arxiv.org/abs/2311.16140v1
• Date: Sat, 4 Nov 2023 14:20:08 GMT
• Title: Adapting Segment Anything Model (SAM) through Prompt-based Learning for Enhanced Protein Identification in Cryo-EM Micrographs
• Authors: Fei He, Zhiyuan Yang, Mingyue Gao, Biplab Poudel, Newgin Sam Ebin Sam Dhas, Rajan Gyawali, Ashwin Dhakal, Jianlin Cheng, Dong Xu
• Abstract summary: cryo-electron microscopy (cryo-EM) remains pivotal in structural biology. Recent AI tools such as Topaz and crYOLO do not fully address the challenges of cryo-EM images. This study explored prompt-based learning to adapt the state-of-the-art image segmentation foundation model Segment Anything Model.
• Score: 16.923131723754192
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Cryo-electron microscopy (cryo-EM) remains pivotal in structural biology, yet the task of protein particle picking, integral for 3D protein structure construction, is laden with manual inefficiencies. While recent AI tools such as Topaz and crYOLO are advancing the field, they do not fully address the challenges of cryo-EM images, including low contrast, complex shapes, and heterogeneous conformations. This study explored prompt-based learning to adapt the state-of-the-art image segmentation foundation model Segment Anything Model (SAM) for cryo-EM. This focus was driven by the desire to optimize model performance with a small number of labeled data without altering pre-trained parameters, aiming for a balance between adaptability and foundational knowledge retention. Through trials with three prompt-based learning strategies, namely head prompt, prefix prompt, and encoder prompt, we observed enhanced performance and reduced computational requirements compared to the fine-tuning approach. This work not only highlights the potential of prompting SAM in protein identification from cryo-EM micrographs but also suggests its broader promise in biomedical image segmentation and object detection.

Related papers

• CryoFM: A Flow-based Foundation Model for Cryo-EM Densities [50.291974465864364]
  We present CryoFM, a foundation model designed as a generative model, learning the distribution of high-quality density maps. Built on flow matching, CryoFM is trained to accurately capture the prior distribution of biomolecular density maps.
  arXiv  Detail & Related papers  (2024-10-11T08:53:58Z)
• Tracing and segmentation of molecular patterns in 3-dimensional cryo-et/em density maps through algorithmic image processing and deep learning-based techniques [0.0]
  dissertation focuses on developing sophisticated computational techniques for tracing actin filaments. Three novel methodologies have been developed: BundleTrac, for tracing bundle-like actin filaments found in Stereocilium, Spaghetti Tracer, for tracing filaments that move individually with loosely cohesive movements, and Struwwel Tracer, for tracing randomly orientated actin filaments in the actin network. The second component of the dissertation introduces a convolutional neural network (CNN) based segmentation model to determine the location of protein secondary structures, such as helices and beta-sheets, in medium-resolution (5-10 Angstrom) 3-dimensional cryo-electron microscopy
  arXiv  Detail & Related papers  (2024-03-26T00:41:54Z)
• xTrimoPGLM: Unified 100B-Scale Pre-trained Transformer for Deciphering the Language of Protein [76.18058946124111]
  We propose a unified protein language model, xTrimoPGLM, to address protein understanding and generation tasks simultaneously. xTrimoPGLM significantly outperforms other advanced baselines in 18 protein understanding benchmarks across four categories. It can also generate de novo protein sequences following the principles of natural ones, and can perform programmable generation after supervised fine-tuning.
  arXiv  Detail & Related papers  (2024-01-11T15:03:17Z)
• Deep learning-based instance segmentation for the precise automated quantification of digital breast cancer immunohistochemistry images [1.8434042562191815]
  We demonstrate the feasibility of using a deep learning-based instance segmentation architecture for the automatic quantification of both nuclear and membrane biomarkers applied to IHC-stained slides. We have collected annotations over samples of HE, ER and Ki-67 (nuclear biomarkers) and HER2 (membrane biomarker) IHC-stained images. We have trained two models, so-called nuclei- and membrane-aware segmentation models, which, once successfully validated, have revealed to be a promising method to segment nuclei instances in IHC-stained images.
  arXiv  Detail & Related papers  (2023-11-22T22:23:47Z)
• Optimizations of Autoencoders for Analysis and Classification of Microscopic In Situ Hybridization Images [68.8204255655161]
  We propose a deep-learning framework to detect and classify areas of microscopic images with similar levels of gene expression. The data we analyze requires an unsupervised learning model for which we employ a type of Artificial Neural Network - Deep Learning Autoencoders.
  arXiv  Detail & Related papers  (2023-04-19T13:45:28Z)
• Semi-Supervised Segmentation of Mitochondria from Electron Microscopy Images Using Spatial Continuity [3.631638087834872]
  We propose a semi-supervised deep learning model that segments mitochondria by leveraging the spatial continuity of their structural, morphological, and contextual information. Our model achieves performance similar to that of state-of-the-art fully supervised models but requires only 20% of their annotated training data.
  arXiv  Detail & Related papers  (2022-06-06T06:52:19Z)
• Learning multi-scale functional representations of proteins from single-cell microscopy data [77.34726150561087]
  We show that simple convolutional networks trained on localization classification can learn protein representations that encapsulate diverse functional information. We also propose a robust evaluation strategy to assess quality of protein representations across different scales of biological function.
  arXiv  Detail & Related papers  (2022-05-24T00:00:07Z)
• Learning Geometrically Disentangled Representations of Protein Folding Simulations [72.03095377508856]
  This work focuses on learning a generative neural network on a structural ensemble of a drug-target protein. Model tasks involve characterizing the distinct structural fluctuations of the protein bound to various drug molecules. Results show that our geometric learning-based method enjoys both accuracy and efficiency for generating complex structural variations.
  arXiv  Detail & Related papers  (2022-05-20T19:38:00Z)
• Towards an Automatic Analysis of CHO-K1 Suspension Growth in Microfluidic Single-cell Cultivation [63.94623495501023]
  We propose a novel Machine Learning architecture, which allows us to infuse a neural deep network with human-powered abstraction on the level of data. Specifically, we train a generative model simultaneously on natural and synthetic data, so that it learns a shared representation, from which a target variable, such as the cell count, can be reliably estimated.
  arXiv  Detail & Related papers  (2020-10-20T08:36:51Z)
• Learning to Segment Anatomical Structures Accurately from One Exemplar [34.287877547953194]
  Methods that permit to produce accurate anatomical structure segmentation without using a large amount of fully annotated training images are highly desirable. We propose Contour Transformer Network (CTN), a one-shot anatomy segmentor including a naturally built-in human-in-the-loop mechanism. We demonstrate that our one-shot learning method significantly outperforms non-learning-based methods and performs competitively to the state-of-the-art fully supervised deep learning approaches.
  arXiv  Detail & Related papers  (2020-07-06T20:27:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.