DINOv2 Rocks Geological Image Analysis: Classification, Segmentation, and Interpretability
- URL: http://arxiv.org/abs/2407.18100v1
- Date: Thu, 25 Jul 2024 15:03:36 GMT
- Title: DINOv2 Rocks Geological Image Analysis: Classification, Segmentation, and Interpretability
- Authors: Florent Brondolo, Samuel Beaussant,
- Abstract summary: This study investigates the interpretability, classification, and segmentation of CT-scan images of rock samples.
We compare various segmentation techniques to evaluate their efficacy, efficiency, and adaptability in geological image analysis.
We find that a LoRA fine-tuned DINOv2 excels in out-of-distribution segmentation and significantly outperforms other methods in multi-class segmentation.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: This study investigates the interpretability, classification, and segmentation of CT-scan images of rock samples, with a particular focus on the application of DINOv2 within Geosciences. We compared various segmentation techniques to evaluate their efficacy, efficiency, and adaptability in geological image analysis. The methods assessed include the Otsu thresholding method, clustering techniques (K-means and fuzzy C-means), a supervised machine learning approach (Random Forest), and deep learning methods (UNet and DINOv2). We tested these methods using ten binary sandstone datasets and three multi-class calcite datasets. To begin, we provide a thorough interpretability analysis of DINOv2's features in the geoscientific context, discussing its suitability and inherent ability to process CT-scanned rock data. In terms of classification, the out-of-the-box DINOv2 demonstrates an impressive capability to perfectly classify rock images, even when the CT scans are out of its original training set. Regarding segmentation, thresholding and unsupervised methods, while fast, perform poorly despite image preprocessing, whereas supervised methods show better results. We underscore the computational demands of deep learning but highlight its minimal intervention, superior generalization, and performance without additional image preprocessing. Additionally, we observe a lack of correlation between a network's depth or the number of parameters and its performance. Our results show that a LoRA fine-tuned DINOv2 excels in out-of-distribution segmentation and significantly outperforms other methods in multi-class segmentation. By systematically comparing these methods, we identify the most efficient strategy for meticulous and laborious segmentation tasks. DINOv2 proves advantageous, achieving segmentations that could be described as "better than ground-truth" against relatively small training sets.
Related papers
- Comparative Evaluation of Traditional and Deep Learning-Based
Segmentation Methods for Spoil Pile Delineation Using UAV Images [0.0]
This study refines and juxtaposes various segmentation approaches, specifically colour-based and morphology-based techniques.
The objective is to enhance and evaluate avenues for object-based analysis for spoil characterisation within the context of mining environments.
Among the diverse segmentation approaches evaluated, the morphology-based deep learning segmentation approach, Segment Anything Model (SAM), exhibited superior performance in comparison to other approaches.
arXiv Detail & Related papers (2024-02-01T02:54:49Z) - Scribble-supervised Cell Segmentation Using Multiscale Contrastive
Regularization [9.849498498869258]
Scribble2Label (S2L) demonstrated that using only a handful of scribbles with self-supervised learning can generate accurate segmentation results without full annotation.
In this work, we employ a novel multiscale contrastive regularization term for S2L.
The main idea is to extract features from intermediate layers of the neural network for contrastive loss so that structures at various scales can be effectively separated.
arXiv Detail & Related papers (2023-06-25T06:00:33Z) - PCA: Semi-supervised Segmentation with Patch Confidence Adversarial
Training [52.895952593202054]
We propose a new semi-supervised adversarial method called Patch Confidence Adrial Training (PCA) for medical image segmentation.
PCA learns the pixel structure and context information in each patch to get enough gradient feedback, which aids the discriminator in convergent to an optimal state.
Our method outperforms the state-of-the-art semi-supervised methods, which demonstrates its effectiveness for medical image segmentation.
arXiv Detail & Related papers (2022-07-24T07:45:47Z) - Learning from Temporal Spatial Cubism for Cross-Dataset Skeleton-based
Action Recognition [88.34182299496074]
Action labels are only available on a source dataset, but unavailable on a target dataset in the training stage.
We utilize a self-supervision scheme to reduce the domain shift between two skeleton-based action datasets.
By segmenting and permuting temporal segments or human body parts, we design two self-supervised learning classification tasks.
arXiv Detail & Related papers (2022-07-17T07:05:39Z) - Nuclei Segmentation with Point Annotations from Pathology Images via
Self-Supervised Learning and Co-Training [44.13451004973818]
We propose a weakly-supervised learning method for nuclei segmentation.
coarse pixel-level labels are derived from the point annotations based on the Voronoi diagram.
A self-supervised visual representation learning method is tailored for nuclei segmentation of pathology images.
arXiv Detail & Related papers (2022-02-16T17:08:44Z) - Learning Fuzzy Clustering for SPECT/CT Segmentation via Convolutional
Neural Networks [5.3123694982708365]
Quantitative bone single-photon emission computed tomography (QBSPECT) has the potential to provide a better quantitative assessment of bone metastasis than planar bone scintigraphy.
The segmentation of anatomical regions-of-interests (ROIs) still relies heavily on the manual delineation by experts.
This work proposes a fast and robust automated segmentation method for partitioning a QBSPECT image into lesion, bone, and background.
arXiv Detail & Related papers (2021-04-17T19:03:52Z) - Dense Contrastive Learning for Self-Supervised Visual Pre-Training [102.15325936477362]
We present dense contrastive learning, which implements self-supervised learning by optimizing a pairwise contrastive (dis)similarity loss at the pixel level between two views of input images.
Compared to the baseline method MoCo-v2, our method introduces negligible computation overhead (only 1% slower)
arXiv Detail & Related papers (2020-11-18T08:42:32Z) - Region Comparison Network for Interpretable Few-shot Image
Classification [97.97902360117368]
Few-shot image classification has been proposed to effectively use only a limited number of labeled examples to train models for new classes.
We propose a metric learning based method named Region Comparison Network (RCN), which is able to reveal how few-shot learning works.
We also present a new way to generalize the interpretability from the level of tasks to categories.
arXiv Detail & Related papers (2020-09-08T07:29:05Z) - MetricUNet: Synergistic Image- and Voxel-Level Learning for Precise CT
Prostate Segmentation via Online Sampling [66.01558025094333]
We propose a two-stage framework, with the first stage to quickly localize the prostate region and the second stage to precisely segment the prostate.
We introduce a novel online metric learning module through voxel-wise sampling in the multi-task network.
Our method can effectively learn more representative voxel-level features compared with the conventional learning methods with cross-entropy or Dice loss.
arXiv Detail & Related papers (2020-05-15T10:37:02Z) - Weakly-Supervised Semantic Segmentation by Iterative Affinity Learning [86.45526827323954]
Weakly-supervised semantic segmentation is a challenging task as no pixel-wise label information is provided for training.
We propose an iterative algorithm to learn such pairwise relations.
We show that the proposed algorithm performs favorably against the state-of-the-art methods.
arXiv Detail & Related papers (2020-02-19T10:32:03Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.