SAM-EM: Real-Time Segmentation for Automated Liquid Phase Transmission Electron Microscopy

• URL: http://arxiv.org/abs/2501.03153v2
• Date: Tue, 04 Nov 2025 19:36:57 GMT
• Title: SAM-EM: Real-Time Segmentation for Automated Liquid Phase Transmission Electron Microscopy
• Abstract summary: We present Segment Anything Model for Electron Microscopy (SAM-EM)<n>SAM-EM is a domain-adapted foundation model that unifies segmentation, tracking, and statistical analysis for LPTEM data.<n>Built on Segment Anything Model 2 (SAM2), SAM-EM is derived through full-model fine-tuning on 46,600 curated LPTEM synthetic video frames.
• Score: 37.81867450552998
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The absence of robust segmentation frameworks for noisy liquid phase transmission electron microscopy (LPTEM) videos prevents reliable extraction of particle trajectories, creating a major barrier to quantitative analysis and to connecting observed dynamics with materials characterization and design. To address this challenge, we present Segment Anything Model for Electron Microscopy (SAM-EM), a domain-adapted foundation model that unifies segmentation, tracking, and statistical analysis for LPTEM data. Built on Segment Anything Model 2 (SAM~2), SAM-EM is derived through full-model fine-tuning on 46,600 curated LPTEM synthetic video frames, substantially improving mask quality and temporal identity stability compared to zero-shot SAM~2 and existing baselines. Beyond segmentation, SAM-EM integrates particle tracking with statistical tools, including mean-squared displacement and particle displacement distribution analysis, providing an end-to-end framework for extracting and interpreting nanoscale dynamics. Crucially, full fine-tuning allows SAM-EM to remain robust under low signal-to-noise conditions, such as those caused by increased liquid sample thickness in LPTEM experiments. By establishing a reliable analysis pipeline, SAM-EM transforms LPTEM into a quantitative single-particle tracking platform and accelerates its integration into data-driven materials discovery and design. Project page: \href{https://github.com/JamaliLab/SAM-EM}{github.com/JamaliLab/SAM-EM}.

Related papers

• Bridging Foundation Models and ASTM Metallurgical Standards for Automated Grain Size Estimation from Microscopy Images [4.3441332321802095]
  We propose an automated pipeline for dense segmentation and grain size estimation.<n>We benchmark this pipeline against a classical convolutional network (U-Net), an adaptive-prompting vision foundation model (MatSAM) and a contemporary vision-language model (Qwen2.5-VL-7B)<n>Experiments across progressively reduced training splits demonstrate exceptional few-shot scalability.
  arXiv  Detail & Related papers  (2026-04-21T01:13:03Z)
• StructSAM: Structure- and Spectrum-Preserving Token Merging for Segment Anything Models [57.674757786328236]
  StructSAM is a resolution-preserving merge-unmerge framework tailored to Segment Anything Model (SAM)<n>We show that StructSAM reduces encoder FLOPs by 25-30% (up to 40%+ with prompt-aware merging) with minor drops in mIoU/Dice.<n>We also provide a spectral graph coarsening view showing that score-guided merging yields bounded Laplacian spectral distortion compared to random or window-restricted baselines.
  arXiv  Detail & Related papers  (2026-03-07T18:30:58Z)
• VesSAM: Efficient Multi-Prompting for Segmenting Complex Vessel [68.24765319399286]
  We present VesSAM, a powerful and efficient framework tailored for 2D vessel segmentation.<n>VesSAM integrates (1) a convolutional adapter to enhance local texture features, (2) a multi-prompt encoder that fuses anatomical prompts, and (3) a lightweight mask decoder to reduce jagged artifacts.<n>VesSAM consistently outperforms state-of-the-art PEFT-based SAM variants by over 10% Dice and 13% IoU.
  arXiv  Detail & Related papers  (2025-11-02T15:47:05Z)
• A Standardized Benchmark for Machine-Learned Molecular Dynamics using Weighted Ensemble Sampling [32.505127447635864]
  We introduce a modular benchmarking framework that systematically evaluates protein MD methods.<n>The framework includes a flexible, lightweight propagator interface that supports arbitrary simulation engines.<n>We contribute a dataset of nine diverse proteins, ranging from 10 to 224 residues, that span a variety of folding complexities.
  arXiv  Detail & Related papers  (2025-10-20T06:02:36Z)
• Automated Machine Learning Pipeline for Training and Analysis Using Large Language Models [0.0]
  We introduce an Automated Machine Learning Pipeline (AMLP) that unifies the entire workflow from dataset creation to model validation.<n>AMLP employs large-language-model agents to assist with electronic-structure code selection, input preparation, and output conversion.<n>It is validated on acridine polymorphs, where, with a straightforward fine-tuning of a foundation model, mean absolute errors of 1.7 meV/atom in energies and 7.0 meV/AA in forces are achieved.
  arXiv  Detail & Related papers  (2025-09-25T22:05:20Z)
• Zero-shot Shape Classification of Nanoparticles in SEM Images using Vision Foundation Models [0.9466841964978984]
  Conventional deep learning methods for shape classification require extensive labeled datasets and computationally demanding training.<n>In this study, we introduce a zero-shot classification pipeline that leverages two vision foundation models.<n>We achieve high-precision shape classification across three morphologically diverse nanoparticle datasets.
  arXiv  Detail & Related papers  (2025-08-05T09:03:56Z)
• DINOv2-powered Few-Shot Semantic Segmentation: A Unified Framework via Cross-Model Distillation and 4D Correlation Mining [30.564216896513596]
  Few-shot semantic segmentation has gained increasing interest due to its generalization capability. Recent approaches have turned to foundation models to enhance representation transferability. We propose FS-DINO, with only DINOv2's encoder and a lightweight segmenter.
  arXiv  Detail & Related papers  (2025-04-22T07:47:06Z)
• CRISP: A Framework for Cryo-EM Image Segmentation and Processing with Conditional Random Field [0.0]
  We present a pipeline that automatically generates high-quality segmentation maps from cryo-EM data. Our modular framework enables the selection of various segmentation models and loss functions. When trained on a limited set of micrographs, our approach achieves over 90% accuracy, recall, precision, Intersection over Union (IoU) and F1-score on synthetic data.
  arXiv  Detail & Related papers  (2025-02-12T10:44:45Z)
• Real-Time Anomaly Detection with Synthetic Anomaly Monitoring (SAM) [2.055524866851853]
  Anomaly detection is essential for identifying rare and significant events across diverse domains such as finance, cybersecurity, and network monitoring.<n>This paper presents Synthetic Anomaly Monitoring (SAM), an innovative approach that applies synthetic control methods from causal inference to improve the accuracy and interpretability of anomaly detection processes.
  arXiv  Detail & Related papers  (2025-01-30T15:15:17Z)
• PolSAM: Polarimetric Scattering Mechanism Informed Segment Anything Model [83.35198885088093]
  PolSAR data presents unique challenges due to its rich and complex characteristics.<n>Existing data representations, such as complex-valued data, polarimetric features, and amplitude images, are widely used.<n>Most feature extraction networks for PolSAR are small, limiting their ability to capture features effectively.<n>We propose the Polarimetric Scattering Mechanism-Informed SAM (PolSAM), an enhanced Segment Anything Model (SAM) that integrates domain-specific scattering characteristics and a novel prompt generation strategy.
  arXiv  Detail & Related papers  (2024-12-17T09:59:53Z)
• A versatile machine learning workflow for high-throughput analysis of supported metal catalyst particles [0.0]
  We introduce a novel two-stage artificial intelligence (AI)-driven workflow for nanoparticles analysis. This methodology was applied to transmission electron microscopy (TEM) and scanning TEM (STEM) images of heterogeneous catalysts. The model's performance in detecting and segmenting NPs was validated across diverse catalyst systems.
  arXiv  Detail & Related papers  (2024-10-02T03:50:01Z)
• Adapting Segment Anything Model for Unseen Object Instance Segmentation [70.60171342436092]
  Unseen Object Instance (UOIS) is crucial for autonomous robots operating in unstructured environments. We propose UOIS-SAM, a data-efficient solution for the UOIS task. UOIS-SAM integrates two key components: (i) a Heatmap-based Prompt Generator (HPG) to generate class-agnostic point prompts with precise foreground prediction, and (ii) a Hierarchical Discrimination Network (HDNet) that adapts SAM's mask decoder.
  arXiv  Detail & Related papers  (2024-09-23T19:05:50Z)
• FusionSAM: Latent Space driven Segment Anything Model for Multimodal Fusion and Segmentation [37.74045675588487]
  We introduce SAM into multimodal image segmentation for the first time. We propose a novel framework that combines Latent Space Token Generation (LSTG) and Fusion Mask Prompting (FMP) modules to enhance SAM's multimodal fusion and segmentation capabilities.
  arXiv  Detail & Related papers  (2024-08-26T02:20:55Z)
• DA-Flow: Dual Attention Normalizing Flow for Skeleton-based Video Anomaly Detection [52.74152717667157]
  We propose a lightweight module called Dual Attention Module (DAM) for capturing cross-dimension interaction relationships in-temporal skeletal data. It employs the frame attention mechanism to identify the most significant frames and the skeleton attention mechanism to capture broader relationships across fixed partitions with minimal parameters and flops.
  arXiv  Detail & Related papers  (2024-06-05T06:18:03Z)
• PTQ4SAM: Post-Training Quantization for Segment Anything [28.893095276574893]
  Segment Anything Model (SAM) has achieved impressive performance in many computer vision tasks. However, as a large-scale model, the immense memory and computation costs hinder its practical deployment. We propose a post-training quantization framework for Segment Anything Model, namely PTQ4SAM.
  arXiv  Detail & Related papers  (2024-05-06T03:39:50Z)
• WSI-SAM: Multi-resolution Segment Anything Model (SAM) for histopathology whole-slide images [8.179859593451285]
  We present WSI-SAM, enhancing Segment Anything Model (SAM) with precise object segmentation capabilities for histopathology images. To fully exploit pretrained knowledge while minimizing training overhead, we keep SAM frozen, introducing only minimal extra parameters. Our model outperforms SAM by 4.1 and 2.5 percent points on a ductal carcinoma in situ (DCIS) segmentation tasks and breast cancer metastasis segmentation task.
  arXiv  Detail & Related papers  (2024-03-14T10:30:43Z)
• Synthetic location trajectory generation using categorical diffusion models [50.809683239937584]
  Diffusion models (DPMs) have rapidly evolved to be one of the predominant generative models for the simulation of synthetic data. We propose using DPMs for the generation of synthetic individual location trajectories (ILTs) which are sequences of variables representing physical locations visited by individuals.
  arXiv  Detail & Related papers  (2024-02-19T15:57:39Z)
• Appearance-Based Refinement for Object-Centric Motion Segmentation [85.2426540999329]
  We introduce an appearance-based refinement method that leverages temporal consistency in video streams to correct inaccurate flow-based proposals. Our approach involves a sequence-level selection mechanism that identifies accurate flow-predicted masks as exemplars. Its performance is evaluated on multiple video segmentation benchmarks, including DAVIS, YouTube, SegTrackv2, and FBMS-59.
  arXiv  Detail & Related papers  (2023-12-18T18:59:51Z)
• TrackDiffusion: Tracklet-Conditioned Video Generation via Diffusion Models [75.20168902300166]
  We propose TrackDiffusion, a novel video generation framework affording fine-grained trajectory-conditioned motion control. A pivotal component of TrackDiffusion is the instance enhancer, which explicitly ensures inter-frame consistency of multiple objects. generated video sequences by our TrackDiffusion can be used as training data for visual perception models.
  arXiv  Detail & Related papers  (2023-12-01T15:24:38Z)
• Recursive Detection and Analysis of Nanoparticles in Scanning Electron Microscopy Images [0.0]
  We present a computational framework tailored for the precise detection and comprehensive analysis of nanoparticles within scanning electron microscopy (SEM) images. The framework employs the robust image processing capabilities of Python, particularly harnessing libraries such as OpenCV, SciPy, and Scikit-Image. It boasts 97% accuracy in detecting particles across five distinct test images drawn from a SEM nanoparticles dataset.
  arXiv  Detail & Related papers  (2023-08-17T02:08:05Z)
• Segment and Track Anything [57.20918630166862]
  This report presents a framework called Segment And Track Anything (SAMTrack) SAM-Track allows users to precisely and effectively segment and track any object in a video. It can be used across an array of fields, ranging from drone technology, autonomous driving, medical imaging, augmented reality, to biological analysis.
  arXiv  Detail & Related papers  (2023-05-11T04:33:08Z)
• Co-attention Propagation Network for Zero-Shot Video Object Segmentation [91.71692262860323]
  Zero-shot object segmentation (ZS-VOS) aims to segment objects in a video sequence without prior knowledge of these objects. Existing ZS-VOS methods often struggle to distinguish between foreground and background or to keep track of the foreground in complex scenarios. We propose an encoder-decoder-based hierarchical co-attention propagation network (HCPN) capable of tracking and segmenting objects.
  arXiv  Detail & Related papers  (2023-04-08T04:45:48Z)
• Segmenting Moving Objects via an Object-Centric Layered Representation [100.26138772664811]
  We introduce an object-centric segmentation model with a depth-ordered layer representation. We introduce a scalable pipeline for generating synthetic training data with multiple objects. We evaluate the model on standard video segmentation benchmarks.
  arXiv  Detail & Related papers  (2022-07-05T17:59:43Z)
• A Deep Learning Based Automatic Defect Analysis Framework for In-situ TEM Ion Irradiations [0.0]
  We develop an automated TEM video analysis system for microstructural features based on the advanced object detection model called YOLO. Results show that the system can achieve human comparable performance with an F1 score of 0.89 for fast, consistent, and scalable frame-level defect analysis.
  arXiv  Detail & Related papers  (2021-08-19T19:15:44Z)
• Improving the Segmentation of Scanning Probe Microscope Images using Convolutional Neural Networks [0.9236074230806579]
  We develop protocols for the segmentation of images of 2D assemblies of gold nanoparticles formed on silicon surfaces via deposition from an organic solvent. The evaporation of the solvent drives far-from-equilibrium self-organisation of the particles, producing a wide variety of nano- and micro-structured patterns. We show that a segmentation strategy using the U-Net convolutional neural network outperforms traditional automated approaches.
  arXiv  Detail & Related papers  (2020-08-27T20:49:59Z)
• Fast Video Object Segmentation With Temporal Aggregation Network and Dynamic Template Matching [67.02962970820505]
  We introduce "tracking-by-detection" into Video Object (VOS) We propose a new temporal aggregation network and a novel dynamic time-evolving template matching mechanism to achieve significantly improved performance. We achieve new state-of-the-art performance on the DAVIS benchmark without complicated bells and whistles in both speed and accuracy, with a speed of 0.14 second per frame and J&F measure of 75.9% respectively.
  arXiv  Detail & Related papers  (2020-07-11T05:44:16Z)
• A Generative Learning Approach for Spatio-temporal Modeling in Connected Vehicular Network [55.852401381113786]
  This paper proposes LaMI (Latency Model Inpainting), a novel framework to generate a comprehensive-temporal quality framework for wireless access latency of connected vehicles. LaMI adopts the idea from image inpainting and synthesizing and can reconstruct the missing latency samples by a two-step procedure. In particular, it first discovers the spatial correlation between samples collected in various regions using a patching-based approach and then feeds the original and highly correlated samples into a Varienational Autocoder (VAE)
  arXiv  Detail & Related papers  (2020-03-16T03:43:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.