A Multispectral Automated Transfer Technique (MATT) for machine-driven image labeling utilizing the Segment Anything Model (SAM)

• URL: http://arxiv.org/abs/2402.11413v1
• Date: Sun, 18 Feb 2024 01:01:13 GMT
• Title: A Multispectral Automated Transfer Technique (MATT) for machine-driven image labeling utilizing the Segment Anything Model (SAM)
• Authors: James E. Gallagher, Aryav Gogia, Edward J. Oughton
• Abstract summary: This paper outlines a method we call the Multispectral Automated Transfer Technique (MATT) By transposing SAM segmentation masks from RGB images we can automatically segment and label multispectral imagery with high precision and efficiency. This research greatly contributes to the study of multispectral object detection by providing a novel and open-source method to rapidly segment, label, and train multispectral object detection models.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Segment Anything Model (SAM) is drastically accelerating the speed and accuracy of automatically segmenting and labeling large Red-Green-Blue (RGB) imagery datasets. However, SAM is unable to segment and label images outside of the visible light spectrum, for example, for multispectral or hyperspectral imagery. Therefore, this paper outlines a method we call the Multispectral Automated Transfer Technique (MATT). By transposing SAM segmentation masks from RGB images we can automatically segment and label multispectral imagery with high precision and efficiency. For example, the results demonstrate that segmenting and labeling a 2,400-image dataset utilizing MATT achieves a time reduction of 87.8% in developing a trained model, reducing roughly 20 hours of manual labeling, to only 2.4 hours. This efficiency gain is associated with only a 6.7% decrease in overall mean average precision (mAP) when training multispectral models via MATT, compared to a manually labeled dataset. We consider this an acceptable level of precision loss when considering the time saved during training, especially for rapidly prototyping experimental modeling methods. This research greatly contributes to the study of multispectral object detection by providing a novel and open-source method to rapidly segment, label, and train multispectral object detection models with minimal human interaction. Future research needs to focus on applying these methods to (i) space-based multispectral, and (ii) drone-based hyperspectral imagery.

Related papers

• SOOD++: Leveraging Unlabeled Data to Boost Oriented Object Detection [59.868772767818975]
  We propose a simple yet effective Semi-supervised Oriented Object Detection method termed SOOD++. Specifically, we observe that objects from aerial images are usually arbitrary orientations, small scales, and aggregation. Extensive experiments conducted on various multi-oriented object datasets under various labeled settings demonstrate the effectiveness of our method.
  arXiv  Detail & Related papers  (2024-07-01T07:03:51Z)
• GenFace: A Large-Scale Fine-Grained Face Forgery Benchmark and Cross Appearance-Edge Learning [50.7702397913573]
  The rapid advancement of photorealistic generators has reached a critical juncture where the discrepancy between authentic and manipulated images is increasingly indistinguishable. Although there have been a number of publicly available face forgery datasets, the forgery faces are mostly generated using GAN-based synthesis technology. We propose a large-scale, diverse, and fine-grained high-fidelity dataset, namely GenFace, to facilitate the advancement of deepfake detection.
  arXiv  Detail & Related papers  (2024-02-03T03:13:50Z)
• SAM-based instance segmentation models for the automation of structural damage detection [0.0]
  We present a data set for instance segmentation with 1,300 annotated images (640 pixels x 640 pixels), named as M1300, covering bricks, broken bricks, and cracks. We test several leading algorithms for benchmarking, including the latest large-scale model, the prompt-based Segment Anything Model (SAM) We propose two novel methods for automation of SAM execution. The first method involves abandoning the prompt encoder and connecting the SAM encoder to other decoders, while the second method introduces a learnable self-generating prompter.
  arXiv  Detail & Related papers  (2024-01-27T02:00:07Z)
• Multimodal Industrial Anomaly Detection by Crossmodal Feature Mapping [12.442574943138794]
  The paper explores the industrial multimodal Anomaly Detection (AD) task, which exploits point clouds and RGB images to localize anomalies. We introduce a novel light and fast framework that learns to map features from one modality to the other on nominal samples.
  arXiv  Detail & Related papers  (2023-12-07T18:41:21Z)
• A Dual Attentive Generative Adversarial Network for Remote Sensing Image Change Detection [6.906936669510404]
  We propose a dual attentive generative adversarial network for achieving very high-resolution remote sensing image change detection tasks. The DAGAN framework has better performance with 85.01% mean IoU and 91.48% mean F1 score than advanced methods on the LEVIR dataset.
  arXiv  Detail & Related papers  (2023-10-03T08:26:27Z)
• PatchCT: Aligning Patch Set and Label Set with Conditional Transport for Multi-Label Image Classification [48.929583521641526]
  Multi-label image classification is a prediction task that aims to identify more than one label from a given image. This paper introduces the conditional transport theory to bridge the acknowledged gap. We find that by formulating the multi-label classification as a CT problem, we can exploit the interactions between the image and label efficiently.
  arXiv  Detail & Related papers  (2023-07-18T08:37:37Z)
• Revisiting Consistency Regularization for Semi-supervised Change Detection in Remote Sensing Images [60.89777029184023]
  We propose a semi-supervised CD model in which we formulate an unsupervised CD loss in addition to the supervised Cross-Entropy (CE) loss. Experiments conducted on two publicly available CD datasets show that the proposed semi-supervised CD method can reach closer to the performance of supervised CD.
  arXiv  Detail & Related papers  (2022-04-18T17:59:01Z)
• Contrastive Multiview Coding with Electro-optics for SAR Semantic Segmentation [0.6445605125467573]
  We propose multi-modal representation learning for SAR semantic segmentation. Unlike previous studies, our method jointly uses EO imagery, SAR imagery, and a label mask. Several experiments show that our approach is superior to the existing methods in model performance, sample efficiency, and convergence speed.
  arXiv  Detail & Related papers  (2021-08-31T23:55:41Z)
• A Hierarchical Transformation-Discriminating Generative Model for Few Shot Anomaly Detection [93.38607559281601]
  We devise a hierarchical generative model that captures the multi-scale patch distribution of each training image. The anomaly score is obtained by aggregating the patch-based votes of the correct transformation across scales and image regions.
  arXiv  Detail & Related papers  (2021-04-29T17:49:48Z)
• A Comparison of Deep Learning Object Detection Models for Satellite Imagery [0.0]
  We compare the detection accuracy and speed of several state-of-the-art models for the task of detecting oil and gas fracking wells and small cars in commercial electro-optical satellite imagery. For the detection of fracking well pads (50m - 250m), we find single-stage detectors provide superior prediction speed while also matching detection performance of their two and multi-stage counterparts. For detecting small cars, two-stage and multi-stage models provide substantially higher accuracies at the cost of some speed.
  arXiv  Detail & Related papers  (2020-09-10T13:43:14Z)
• Anchor-free Small-scale Multispectral Pedestrian Detection [88.7497134369344]
  We propose a method for effective and efficient multispectral fusion of the two modalities in an adapted single-stage anchor-free base architecture. We aim at learning pedestrian representations based on object center and scale rather than direct bounding box predictions. Results show our method's effectiveness in detecting small-scaled pedestrians.
  arXiv  Detail & Related papers  (2020-08-19T13:13:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.