FaultSeg Swin-UNETR: Transformer-Based Self-Supervised Pretraining Model for Fault Recognition

• URL: http://arxiv.org/abs/2310.17974v2
• Date: Tue, 9 Jan 2024 03:12:23 GMT
• Title: FaultSeg Swin-UNETR: Transformer-Based Self-Supervised Pretraining Model for Fault Recognition
• Authors: Zeren Zhang, Ran Chen, Jinwen Ma
• Abstract summary: This paper introduces an approach to enhance seismic fault recognition through self-supervised pretraining. We have employed the Swin Transformer model as the core network and employed the SimMIM pretraining task to capture unique features related to discontinuities in seismic data. Experimental results demonstrate that our proposed method attains state-of-the-art performance on the Thebe dataset, as measured by the OIS and ODS metrics.
• Score: 13.339333273943842
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper introduces an approach to enhance seismic fault recognition through self-supervised pretraining. Seismic fault interpretation holds great significance in the fields of geophysics and geology. However, conventional methods for seismic fault recognition encounter various issues, including dependence on data quality and quantity, as well as susceptibility to interpreter subjectivity. Currently, automated fault recognition methods proposed based on small synthetic datasets experience performance degradation when applied to actual seismic data. To address these challenges, we have introduced the concept of self-supervised learning, utilizing a substantial amount of relatively easily obtainable unlabeled seismic data for pretraining. Specifically, we have employed the Swin Transformer model as the core network and employed the SimMIM pretraining task to capture unique features related to discontinuities in seismic data. During the fine-tuning phase, inspired by edge detection techniques, we have also refined the structure of the Swin-UNETR model, enabling multiscale decoding and fusion for more effective fault detection. Experimental results demonstrate that our proposed method attains state-of-the-art performance on the Thebe dataset, as measured by the OIS and ODS metrics.

Related papers

• SINDER: Repairing the Singular Defects of DINOv2 [61.98878352956125]
  Vision Transformer models trained on large-scale datasets often exhibit artifacts in the patch token they extract. We propose a novel fine-tuning smooth regularization that rectifies structural deficiencies using only a small dataset.
  arXiv  Detail & Related papers  (2024-07-23T20:34:23Z)
• Reshaping the Online Data Buffering and Organizing Mechanism for Continual Test-Time Adaptation [49.53202761595912]
  Continual Test-Time Adaptation involves adapting a pre-trained source model to continually changing unsupervised target domains. We analyze the challenges of this task: online environment, unsupervised nature, and the risks of error accumulation and catastrophic forgetting. We propose an uncertainty-aware buffering approach to identify and aggregate significant samples with high certainty from the unsupervised, single-pass data stream.
  arXiv  Detail & Related papers  (2024-07-12T15:48:40Z)
• The Significance of Latent Data Divergence in Predicting System Degradation [1.2058600649065616]
  Condition-Based Maintenance is pivotal in enabling the early detection of potential failures in engineering systems. We introduce a novel methodology grounded in the analysis of statistical similarity within latent data from system components. We infer the similarity between systems by evaluating the divergence of these priors, offering a nuanced understanding of individual system behaviors.
  arXiv  Detail & Related papers  (2024-06-13T11:41:20Z)
• Federated Learning with Anomaly Detection via Gradient and Reconstruction Analysis [2.28438857884398]
  We introduce a novel framework that synergizes gradient-based analysis with autoencoder-driven data reconstruction to detect and mitigate poisoned data with unprecedented precision. Our method outperforms existing solutions by 15% in anomaly detection accuracy while maintaining a minimal false positive rate. Our work paves the way for future advancements in distributed learning security.
  arXiv  Detail & Related papers  (2024-03-15T03:54:45Z)
• Self-Supervised Graph Transformer for Deepfake Detection [1.8133635752982105]
  Deepfake detection methods have shown promising results in recognizing forgeries within a given dataset. Deepfake detection system must remain impartial to forgery types, appearance, and quality for guaranteed generalizable detection performance. This study introduces a deepfake detection framework, leveraging a self-supervised pre-training model that delivers exceptional generalization ability.
  arXiv  Detail & Related papers  (2023-07-27T17:22:41Z)
• Cluster-level pseudo-labelling for source-free cross-domain facial expression recognition [94.56304526014875]
  We propose the first Source-Free Unsupervised Domain Adaptation (SFUDA) method for Facial Expression Recognition (FER) Our method exploits self-supervised pretraining to learn good feature representations from the target data. We validate the effectiveness of our method in four adaptation setups, proving that it consistently outperforms existing SFUDA methods when applied to FER.
  arXiv  Detail & Related papers  (2022-10-11T08:24:50Z)
• Toward Certified Robustness Against Real-World Distribution Shifts [65.66374339500025]
  We train a generative model to learn perturbations from data and define specifications with respect to the output of the learned model. A unique challenge arising from this setting is that existing verifiers cannot tightly approximate sigmoid activations. We propose a general meta-algorithm for handling sigmoid activations which leverages classical notions of counter-example-guided abstraction refinement.
  arXiv  Detail & Related papers  (2022-06-08T04:09:13Z)
• Improving robustness of jet tagging algorithms with adversarial training [56.79800815519762]
  We investigate the vulnerability of flavor tagging algorithms via application of adversarial attacks. We present an adversarial training strategy that mitigates the impact of such simulated attacks.
  arXiv  Detail & Related papers  (2022-03-25T19:57:19Z)
• Learning class prototypes from Synthetic InSAR with Vision Transformers [2.41710192205034]
  Detection of early signs of volcanic unrest is critical for assessing volcanic hazard. We propose a novel deep learning methodology that exploits a rich source of synthetically generated interferograms. We report detection accuracy that surpasses the state of the art on volcanic unrest detection.
  arXiv  Detail & Related papers  (2022-01-09T14:03:00Z)
• Unsupervised Domain Adaptation for Speech Recognition via Uncertainty Driven Self-Training [55.824641135682725]
  Domain adaptation experiments using WSJ as a source domain and TED-LIUM 3 as well as SWITCHBOARD show that up to 80% of the performance of a system trained on ground-truth data can be recovered.
  arXiv  Detail & Related papers  (2020-11-26T18:51:26Z)
• Data-Driven Failure Prediction in Brittle Materials: A Phase-Field Based Machine Learning Framework [1.3858051019755282]
  Failure in brittle materials led by micro- to macro-cracks under repetitive or increasing loads is often catastrophic. We develop a supervised machine learning (ML) framework to predict failure in an isothermal, linear elastic and isotropic phase-field model. Results indicate that the proposed framework is capable of predicting failure with acceptable accuracy even in the presence of high noise levels.
  arXiv  Detail & Related papers  (2020-03-24T17:13:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.