SEM-CLIP: Precise Few-Shot Learning for Nanoscale Defect Detection in Scanning Electron Microscope Image

• URL: http://arxiv.org/abs/2502.14884v1
• Date: Sat, 15 Feb 2025 14:06:48 GMT
• Title: SEM-CLIP: Precise Few-Shot Learning for Nanoscale Defect Detection in Scanning Electron Microscope Image
• Authors: Qian Jin, Yuqi Jiang, Xudong Lu, Yumeng Liu, Yining Chen, Dawei Gao, Qi Sun, Cheng Zhuo,
• Abstract summary: We propose a novel few-shot learning approach, SEM-CLIP, for accurate defect classification and segmentation.<n>We employ text prompts enriched with domain knowledge as prior information to assist in precise analysis.<n> SEM-CLIP requires little annotated data, substantially reducing labor demands in the semiconductor industry.
• Score: 27.17821452832534
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In the field of integrated circuit manufacturing, the detection and classification of nanoscale wafer defects are critical for subsequent root cause analysis and yield enhancement. The complex background patterns observed in scanning electron microscope (SEM) images and the diverse textures of the defects pose significant challenges. Traditional methods usually suffer from insufficient data, labels, and poor transferability. In this paper, we propose a novel few-shot learning approach, SEM-CLIP, for accurate defect classification and segmentation. SEM-CLIP customizes the Contrastive Language-Image Pretraining (CLIP) model to better focus on defect areas and minimize background distractions, thereby enhancing segmentation accuracy. We employ text prompts enriched with domain knowledge as prior information to assist in precise analysis. Additionally, our approach incorporates feature engineering with textual guidance to categorize defects more effectively. SEM-CLIP requires little annotated data, substantially reducing labor demands in the semiconductor industry. Extensive experimental validation demonstrates that our model achieves impressive classification and segmentation results under few-shot learning scenarios.

Related papers

• Crane: Context-Guided Prompt Learning and Attention Refinement for Zero-Shot Anomaly Detections [50.343419243749054]
  Anomaly Detection (AD) involves identifying deviations from normal data distributions. We propose a novel approach that conditions the prompts of the text encoder based on image context extracted from the vision encoder. Our method achieves state-of-the-art performance, improving performance by 2% to 29% across different metrics on 14 datasets.
  arXiv  Detail & Related papers  (2025-04-15T10:42:25Z)
• What Really Matters for Learning-based LiDAR-Camera Calibration [50.2608502974106]
  This paper revisits the development of learning-based LiDAR-Camera calibration.<n>We identify the critical limitations of regression-based methods with the widely used data generation pipeline.<n>We also investigate how the input data format and preprocessing operations impact network performance.
  arXiv  Detail & Related papers  (2025-01-28T14:12:32Z)
• Improving the Precision of CNNs for Magnetic Resonance Spectral Modeling [4.638569496003459]
  Using machine learning to predict MRS-related quantities offers avenues around this problem, but deep learning models bring their own challenges. This work highlights why more comprehensive error characterization is important and how to improve the precision of CNNs for spectral modeling.
  arXiv  Detail & Related papers  (2024-09-10T16:02:12Z)
• Improving Vision Anomaly Detection with the Guidance of Language Modality [64.53005837237754]
  This paper tackles the challenges for vision modality from a multimodal point of view. We propose Cross-modal Guidance (CMG) to tackle the redundant information issue and sparse space issue. To learn a more compact latent space for the vision anomaly detector, CMLE learns a correlation structure matrix from the language modality.
  arXiv  Detail & Related papers  (2023-10-04T13:44:56Z)
• CINFormer: Transformer network with multi-stage CNN feature injection for surface defect segmentation [73.02218479926469]
  We propose a transformer network with multi-stage CNN feature injection for surface defect segmentation. CINFormer presents a simple yet effective feature integration mechanism that injects the multi-level CNN features of the input image into different stages of the transformer network in the encoder. In addition, CINFormer presents a Top-K self-attention module to focus on tokens with more important information about the defects.
  arXiv  Detail & Related papers  (2023-09-22T06:12:02Z)
• Automated Semiconductor Defect Inspection in Scanning Electron Microscope Images: a Systematic Review [4.493547775253646]
  Machine learning algorithms can be trained to accurately classify and locate defects in semiconductor samples. Convolutional neural networks have proved to be particularly useful in this regard. This systematic review provides a comprehensive overview of the state of automated semiconductor defect inspection on SEM images.
  arXiv  Detail & Related papers  (2023-08-16T13:59:43Z)
• Seamless Iterative Semi-Supervised Correction of Imperfect Labels in Microscopy Images [57.42492501915773]
  In-vitro tests are an alternative to animal testing for the toxicity of medical devices. Human fatigue plays a role in error making, making the use of deep learning appealing. We propose Seamless Iterative Semi-Supervised correction of Imperfect labels (SISSI) Our method successfully provides an adaptive early learning correction technique for object detection.
  arXiv  Detail & Related papers  (2022-08-05T18:52:20Z)
• Negligible effect of brain MRI data preprocessing for tumor segmentation [36.89606202543839]
  We conduct experiments on three publicly available datasets and evaluate the effect of different preprocessing steps in deep neural networks. Our results demonstrate that most popular standardization steps add no value to the network performance. We suggest that image intensity normalization approaches do not contribute to model accuracy because of the reduction of signal variance with image standardization.
  arXiv  Detail & Related papers  (2022-04-11T17:29:36Z)
• Digital Fingerprinting of Microstructures [44.139970905896504]
  Finding efficient means of fingerprinting microstructural information is a critical step towards harnessing data-centric machine learning approaches. Here, we consider microstructure classification and utilise the resulting features over a range of related machine learning tasks. In particular, methods that leverage transfer learning with convolutional neural networks (CNNs), pretrained on the ImageNet dataset, are generally shown to outperform other methods.
  arXiv  Detail & Related papers  (2022-03-25T15:40:44Z)
• Calibrating the Dice loss to handle neural network overconfidence for biomedical image segmentation [2.6465053740712157]
  The Dice similarity coefficient (DSC) is a widely used metric and loss function for biomedical image segmentation. In this study, we identify poor calibration as an emerging challenge of deep learning based biomedical image segmentation. We provide a simple yet effective extension of the DSC loss, named the DSC++ loss, that selectively modulates the penalty associated with overconfident, incorrect predictions.
  arXiv  Detail & Related papers  (2021-10-31T16:02:02Z)
• Performance, Successes and Limitations of Deep Learning Semantic Segmentation of Multiple Defects in Transmission Electron Micrographs [9.237363938772479]
  We perform semantic segmentation of defect types in electron microscopy images of irradiated FeCrAl alloys using a deep learning Mask Regional Convolutional Neural Network (Mask R-CNN) model. We conduct an in-depth analysis of key model performance statistics, with a focus on quantities such as predicted distributions of defect shapes, defect sizes, and defect areal densities. Overall, we find that the current model is a fast, effective tool for automatically characterizing and quantifying multiple defect types in microscopy images.
  arXiv  Detail & Related papers  (2021-10-15T17:57:59Z)
• Label Cleaning Multiple Instance Learning: Refining Coarse Annotations on Single Whole-Slide Images [83.7047542725469]
  Annotating cancerous regions in whole-slide images (WSIs) of pathology samples plays a critical role in clinical diagnosis, biomedical research, and machine learning algorithms development. We present a method, named Label Cleaning Multiple Instance Learning (LC-MIL), to refine coarse annotations on a single WSI without the need of external training data. Our experiments on a heterogeneous WSI set with breast cancer lymph node metastasis, liver cancer, and colorectal cancer samples show that LC-MIL significantly refines the coarse annotations, outperforming the state-of-the-art alternatives, even while learning from a single slide.
  arXiv  Detail & Related papers  (2021-09-22T15:06:06Z)
• Learning-based Defect Recognition for Quasi-Periodic Microscope Images [0.0]
  We propose a semi-supervised machine learning method that assists in the detection of lattice defects from atomic resolution microscope images. It involves a convolutional neural network that classifies image patches as defective or non-defective, a graph-based that chooses one non-defective patch as a model, and finally an automatically generated convolutional filter bank. The algorithm is tested on III-V/Si crystalline materials and successfully evaluated against different metrics, showing promising results even for extremely small training data sets.
  arXiv  Detail & Related papers  (2020-07-02T18:00:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.