Detecting immune cells with label-free two-photon autofluorescence and deep learning

• URL: http://arxiv.org/abs/2506.14449v1
• Date: Tue, 17 Jun 2025 12:14:02 GMT
• Title: Detecting immune cells with label-free two-photon autofluorescence and deep learning
• Authors: Lucas Kreiss, Amey Chaware, Maryam Roohian, Sarah Lemire, Oana-Maria Thoma, Birgitta Carlé, Maximilian Waldner, Sebastian Schürmann, Oliver Friedrich, Roarke Horstmeyer,
• Abstract summary: We trained a convolutional neural network (CNN) to classify cell types based on this label-free AF as input.<n>A low-complexity squeezeNet architecture was able to achieve reliable immune cell classification results.<n>In the future, such predictive DL models could directly detect specific immune cells in unstained images.
• Score: 2.096497520824982
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Label-free imaging has gained broad interest because of its potential to omit elaborate staining procedures which is especially relevant for in vivo use. Label-free multiphoton microscopy (MPM), for instance, exploits two-photon excitation of natural autofluorescence (AF) from native, metabolic proteins, making it ideal for in vivo endomicroscopy. Deep learning (DL) models have been widely used in other optical imaging technologies to predict specific target annotations and thereby digitally augment the specificity of these label-free images. However, this computational specificity has only rarely been implemented for MPM. In this work, we used a data set of label-free MPM images from a series of different immune cell types (5,075 individual cells for binary classification in mixed samples and 3,424 cells for a multi-class classification task) and trained a convolutional neural network (CNN) to classify cell types based on this label-free AF as input. A low-complexity squeezeNet architecture was able to achieve reliable immune cell classification results (0.89 ROC-AUC, 0.95 PR-AUC, for binary classification in mixed samples; 0.689 F1 score, 0.697 precision, 0.748 recall, and 0.683 MCC for six-class classification in isolated samples). Perturbation tests confirmed that the model is not confused by extracellular environment and that both input AF channels (NADH and FAD) are about equally important to the classification. In the future, such predictive DL models could directly detect specific immune cells in unstained images and thus, computationally improve the specificity of label-free MPM which would have great potential for in vivo endomicroscopy.

Related papers

• MIPHEI-ViT: Multiplex Immunofluorescence Prediction from H&E Images using ViT Foundation Models [2.2802322828460864]
  We introduce MIPHEI, a U-Net-inspired architecture that integrates state-of-the-art ViT foundation models as encoders to predict mIF signals from H&E images.<n>We train our model using the publicly available ORION dataset of restained H&E and mIF images from colorectal cancer tissue.<n>MIPHEI achieves accurate cell-type classification from H&E alone, with F1 scores of 0.88 for Pan-CK, 0.57 for CD3e, 0.56 for SMA, 0.36 for CD68, and 0.30 for CD20.
  arXiv  Detail & Related papers  (2025-05-15T13:42:48Z)
• DiffKillR: Killing and Recreating Diffeomorphisms for Cell Annotation in Dense Microscopy Images [105.46086313858062]
  We introduce DiffKillR, a novel framework that reframes cell annotation as the combination of archetype matching and image registration tasks.<n>DiffKillR efficiently propagates annotations across large microscopy images, reducing the need for extensive manual labeling.<n>We will discuss the theoretical properties of DiffKillR and validate it on three microscopy tasks, demonstrating its advantages over existing supervised, semi-supervised, and unsupervised methods.
  arXiv  Detail & Related papers  (2024-10-04T00:38:29Z)
• Semi-supervised variational autoencoder for cell feature extraction in multiplexed immunofluorescence images [40.234346302444536]
  We propose a deep learning-based cell feature extraction model using a variational autoencoder with supervision. We perform cell phenotype classification using a cohort of more than 44,000 multiplexed immunofluorescence cell image patches extracted across 1,093 tissue microarray cores of breast cancer patients.
  arXiv  Detail & Related papers  (2024-06-22T04:32:50Z)
• MMIL: A novel algorithm for disease associated cell type discovery [58.044870442206914]
  Single-cell datasets often lack individual cell labels, making it challenging to identify cells associated with disease. We introduce Mixture Modeling for Multiple Learning Instance (MMIL), an expectation method that enables the training and calibration of cell-level classifiers.
  arXiv  Detail & Related papers  (2024-06-12T15:22:56Z)
• Immunocto: a massive immune cell database auto-generated for histopathology [0.0]
  We create Immunocto, a massive, multi-million automatically generated database of 6,848,454 human cells and objects.<n>For each cell, we provide a 64$times$64 pixels$2$ H&E image at $mathbf40times$ magnification, along with a binary mask of the nucleus and a label.<n>Deep learning models trained on Immunocto result in state-of-the-art performance for lymphocyte detection.
  arXiv  Detail & Related papers  (2024-06-03T17:03:58Z)
• Practical Guidelines for Cell Segmentation Models Under Optical Aberrations in Microscopy [14.042884268397058]
  This study evaluates cell image segmentation models under optical aberrations from fluorescence and bright field microscopy. We train and test several segmentation models, including the Otsu threshold method and Mask R-CNN with different network heads. In contrast, Cellpose 2.0 proves effective for complex cell images under similar conditions.
  arXiv  Detail & Related papers  (2024-04-12T15:45:26Z)
• Neural Cellular Automata for Lightweight, Robust and Explainable Classification of White Blood Cell Images [40.347953893940044]
  We introduce a novel approach for white blood cell classification based on neural cellular automata (NCA) Our NCA-based method is significantly smaller in terms of parameters and exhibits robustness to domain shifts. Our results demonstrate that NCA can be used for image classification, and they address key challenges of conventional methods.
  arXiv  Detail & Related papers  (2024-04-08T14:59:53Z)
• Interpretable cancer cell detection with phonon microscopy using multi-task conditional neural networks for inter-batch calibration [39.759100498329275]
  We present a conditional neural network framework to simultaneously achieve inter-batch calibration. We validate our approach by training and validating on different experimental batches. We extend our model to reconstruct denoised signals, enabling physical interpretation of salient features indicating disease state.
  arXiv  Detail & Related papers  (2024-03-26T12:20:10Z)
• Style transfer between Microscopy and Magnetic Resonance Imaging via Generative Adversarial Network in small sample size settings [45.62331048595689]
  Cross-modal augmentation of Magnetic Resonance Imaging (MRI) and microscopic imaging based on the same tissue samples is promising.<n>We tested a method for generating microscopic histological images from MRI scans of the corpus callosum using conditional generative adversarial network (cGAN) architecture.
  arXiv  Detail & Related papers  (2023-10-16T13:58:53Z)
• Corneal endothelium assessment in specular microscopy images with Fuchs' dystrophy via deep regression of signed distance maps [48.498376125522114]
  This paper proposes a UNet-based segmentation approach that requires minimal post-processing. It achieves reliable CE morphometric assessment and guttae identification across all degrees of Fuchs' dystrophy.
  arXiv  Detail & Related papers  (2022-10-13T15:34:20Z)
• Automated Phenotyping via Cell Auto Training (CAT) on the Cell DIVE Platform [0.5599792629509229]
  We present a method for automatic cell classification in tissue samples using an automated training set from multiplexed immunofluorescence images. The method utilizes multiple markers stained in situ on a single tissue section on a robust hyperplex immunofluorescence platform.
  arXiv  Detail & Related papers  (2020-07-18T16:45:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.