Leveraging Computational Pathology AI for Noninvasive Optical Imaging Analysis Without Retraining

• URL: http://arxiv.org/abs/2411.11613v2
• Date: Tue, 19 Nov 2024 15:36:38 GMT
• Title: Leveraging Computational Pathology AI for Noninvasive Optical Imaging Analysis Without Retraining
• Authors: Danny Barash, Emilie Manning, Aidan Van Vleck, Omri Hirsch, Kyi Lei Aye, Jingxi Li, Philip O. Scumpia, Aydogan Ozcan, Sumaira Aasi, Kerri E. Rieger, Kavita Y. Sarin, Oren Freifeld, Yonatan Winetraub,
• Abstract summary: Noninvasive optical imaging modalities can probe patient's tissue in 3D and over time generate gigabytes of clinically relevant data per sample. There is a need for AI models to analyze this data and assist clinical workflow. In this paper we introduce FoundationShift, a method to apply any AI model from computational pathology without retraining.
• Score: 3.6835809728620634
• License:
• Abstract: Noninvasive optical imaging modalities can probe patient's tissue in 3D and over time generate gigabytes of clinically relevant data per sample. There is a need for AI models to analyze this data and assist clinical workflow. The lack of expert labelers and the large dataset required (>100,000 images) for model training and tuning are the main hurdles in creating foundation models. In this paper we introduce FoundationShift, a method to apply any AI model from computational pathology without retraining. We show our method is more accurate than state of the art models (SAM, MedSAM, SAM-Med2D, CellProfiler, Hover-Net, PLIP, UNI and ChatGPT), with multiple imaging modalities (OCT and RCM). This is achieved without the need for model retraining or fine-tuning. Applying our method to noninvasive in vivo images could enable physicians to readily incorporate optical imaging modalities into their clinical practice, providing real time tissue analysis and improving patient care.

Related papers

• Benchmarking Embedding Aggregation Methods in Computational Pathology: A Clinical Data Perspective [32.93871326428446]
  Recent advances in artificial intelligence (AI) are revolutionizing medical imaging and computational pathology. A constant challenge in the analysis of digital Whole Slide Images (WSIs) is the problem of aggregating tens of thousands of tile-level image embeddings to a slide-level representation. This study conducts a benchmarking analysis of ten slide-level aggregation techniques across nine clinically relevant tasks.
  arXiv  Detail & Related papers  (2024-07-10T17:00:57Z)
• Towards a clinically accessible radiology foundation model: open-access and lightweight, with automated evaluation [113.5002649181103]
  Training open-source small multimodal models (SMMs) to bridge competency gaps for unmet clinical needs in radiology. For training, we assemble a large dataset of over 697 thousand radiology image-text pairs. For evaluation, we propose CheXprompt, a GPT-4-based metric for factuality evaluation, and demonstrate its parity with expert evaluation. The inference of LlaVA-Rad is fast and can be performed on a single V100 GPU in private settings, offering a promising state-of-the-art tool for real-world clinical applications.
  arXiv  Detail & Related papers  (2024-03-12T18:12:02Z)
• LVM-Med: Learning Large-Scale Self-Supervised Vision Models for Medical Imaging via Second-order Graph Matching [59.01894976615714]
  We introduce LVM-Med, the first family of deep networks trained on large-scale medical datasets. We have collected approximately 1.3 million medical images from 55 publicly available datasets. LVM-Med empirically outperforms a number of state-of-the-art supervised, self-supervised, and foundation models.
  arXiv  Detail & Related papers  (2023-06-20T22:21:34Z)
• Vision-Language Modelling For Radiological Imaging and Reports In The Low Data Regime [70.04389979779195]
  This paper explores training medical vision-language models (VLMs) where the visual and language inputs are embedded into a common space. We explore several candidate methods to improve low-data performance, including adapting generic pre-trained models to novel image and text domains. Using text-to-image retrieval as a benchmark, we evaluate the performance of these methods with variable sized training datasets of paired chest X-rays and radiological reports.
  arXiv  Detail & Related papers  (2023-03-30T18:20:00Z)
• SYNTA: A novel approach for deep learning-based image analysis in muscle histopathology using photo-realistic synthetic data [2.1616289178832666]
  We introduce SYNTA (synthetic data) as a novel approach for the generation of synthetic, photo-realistic, and highly complex biomedical images as training data. We demonstrate that it is possible to perform robust and expert-level segmentation tasks on previously unseen real-world data, without the need for manual annotations.
  arXiv  Detail & Related papers  (2022-07-29T12:50:32Z)
• SyntheX: Scaling Up Learning-based X-ray Image Analysis Through In Silico Experiments [12.019996672009375]
  We show that creating realistic simulated images from human models is a viable alternative to large-scale in situ data collection. Because synthetic generation of training data from human-based models scales easily, we find that our model transfer paradigm for X-ray image analysis, which we refer to as SyntheX, can even outperform real data-trained models.
  arXiv  Detail & Related papers  (2022-06-13T13:08:41Z)
• Ultrasound Signal Processing: From Models to Deep Learning [64.56774869055826]
  Medical ultrasound imaging relies heavily on high-quality signal processing to provide reliable and interpretable image reconstructions. Deep learning based methods, which are optimized in a data-driven fashion, have gained popularity. A relatively new paradigm combines the power of the two: leveraging data-driven deep learning, as well as exploiting domain knowledge.
  arXiv  Detail & Related papers  (2022-04-09T13:04:36Z)
• Self-Supervised Representation Learning using Visual Field Expansion on Digital Pathology [7.568373895297608]
  A key challenge in the analysis of such images is their size, which can run into the gigapixels. We propose a novel generative framework that can learn powerful representations for such tiles by learning to plausibly expand their visual field. Our model learns to generate different tissue types with fine details, while simultaneously learning powerful representations that can be used for different clinical endpoints.
  arXiv  Detail & Related papers  (2021-09-07T19:20:01Z)
• A parameter refinement method for Ptychography based on Deep Learning concepts [55.41644538483948]
  coarse parametrisation in propagation distance, position errors and partial coherence frequently menaces the experiment viability. A modern Deep Learning framework is used to correct autonomously the setup incoherences, thus improving the quality of a ptychography reconstruction. We tested our system on both synthetic datasets and also on real data acquired at the TwinMic beamline of the Elettra synchrotron facility.
  arXiv  Detail & Related papers  (2021-05-18T10:15:17Z)
• Data-driven generation of plausible tissue geometries for realistic photoacoustic image synthesis [53.65837038435433]
  Photoacoustic tomography (PAT) has the potential to recover morphological and functional tissue properties. We propose a novel approach to PAT data simulation, which we refer to as "learning to simulate" We leverage the concept of Generative Adversarial Networks (GANs) trained on semantically annotated medical imaging data to generate plausible tissue geometries.
  arXiv  Detail & Related papers  (2021-03-29T11:30:18Z)
• Democratizing Artificial Intelligence in Healthcare: A Study of Model Development Across Two Institutions Incorporating Transfer Learning [8.043077408518826]
  Transfer learning (TL) allows a fully trained model from one institution to be fine-tuned by another institution using a much small local dataset. This report describes the challenges, methodology, and benefits of TL within the context of developing an AI model for a basic use-case.
  arXiv  Detail & Related papers  (2020-09-25T21:12:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.