Designing a Hybrid Neural System to Learn Real-world Crack Segmentation from Fractal-based Simulation

• URL: http://arxiv.org/abs/2309.09637v1
• Date: Mon, 18 Sep 2023 10:13:03 GMT
• Title: Designing a Hybrid Neural System to Learn Real-world Crack Segmentation from Fractal-based Simulation
• Authors: Achref Jaziri, Martin Mundt, Andres Fernandez Rodriguez, Visvanathan Ramesh
• Abstract summary: We introduce a high-fidelity crack graphics simulator based on fractals and a corresponding fully-annotated crack dataset. We then complement the latter with a system that learns generalizable representations from simulation. Finally, we empirically highlight how different design choices are symbiotic in bridging the simulation to real gap, and ultimately demonstrate that our introduced system can effectively handle real-world crack segmentation.
• Score: 7.0156884721768575
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Identification of cracks is essential to assess the structural integrity of concrete infrastructure. However, robust crack segmentation remains a challenging task for computer vision systems due to the diverse appearance of concrete surfaces, variable lighting and weather conditions, and the overlapping of different defects. In particular recent data-driven methods struggle with the limited availability of data, the fine-grained and time-consuming nature of crack annotation, and face subsequent difficulty in generalizing to out-of-distribution samples. In this work, we move past these challenges in a two-fold way. We introduce a high-fidelity crack graphics simulator based on fractals and a corresponding fully-annotated crack dataset. We then complement the latter with a system that learns generalizable representations from simulation, by leveraging both a pointwise mutual information estimate along with adaptive instance normalization as inductive biases. Finally, we empirically highlight how different design choices are symbiotic in bridging the simulation to real gap, and ultimately demonstrate that our introduced system can effectively handle real-world crack segmentation.

Related papers

• Boosting Semi-Supervised Scene Text Recognition via Viewing and Summarizing [71.29488677105127]
  Existing scene text recognition (STR) methods struggle to recognize challenging texts, especially for artistic and severely distorted characters. We propose a contrastive learning-based STR framework by leveraging synthetic and real unlabeled data without any human cost. Our method achieves SOTA performance (94.7% and 70.9% average accuracy on common benchmarks and Union14M-Benchmark.
  arXiv  Detail & Related papers  (2024-11-23T15:24:47Z)
• Fast and Reliable Probabilistic Reflectometry Inversion with Prior-Amortized Neural Posterior Estimation [73.81105275628751]
  Finding all structures compatible with reflectometry data is computationally prohibitive for standard algorithms. We address this lack of reliability with a probabilistic deep learning method that identifies all realistic structures in seconds. Our method, Prior-Amortized Neural Posterior Estimation (PANPE), combines simulation-based inference with novel adaptive priors.
  arXiv  Detail & Related papers  (2024-07-26T10:29:16Z)
• A spatiotemporal deep learning framework for prediction of crack dynamics in heterogeneous solids: efficient mapping of concrete microstructures to its fracture properties [0.0]
  A deep learning framework is capable of 2D full-field prediction of fracture in concrete mesostructures. A convolutional neural network is developed which can predict the averaged stress-strain curve of mesostructures. The UNet modeling framework, which comprises an encoder-decoder section with skip connections, is used as the deep learning surrogate model.
  arXiv  Detail & Related papers  (2024-07-22T14:28:46Z)
• DiffusionNOCS: Managing Symmetry and Uncertainty in Sim2Real Multi-Modal Category-level Pose Estimation [20.676510832922016]
  We propose a probabilistic model that relies on diffusion to estimate dense canonical maps crucial for recovering partial object shapes. We introduce critical components to enhance performance by leveraging the strength of the diffusion models with multi-modal input representations. Despite being trained solely on our generated synthetic data, our approach achieves state-of-the-art performance and unprecedented generalization qualities.
  arXiv  Detail & Related papers  (2024-02-20T01:48:33Z)
• Video Anomaly Detection via Spatio-Temporal Pseudo-Anomaly Generation : A Unified Approach [49.995833831087175]
  This work proposes a novel method for generating generic Video-temporal PAs by inpainting a masked out region of an image. In addition, we present a simple unified framework to detect real-world anomalies under the OCC setting. Our method performs on par with other existing state-of-the-art PAs generation and reconstruction based methods under the OCC setting.
  arXiv  Detail & Related papers  (2023-11-27T13:14:06Z)
• A Comprehensive Augmentation Framework for Anomaly Detection [1.6114012813668932]
  This paper analyzes crucial traits of simulated anomalies that contribute to the training of reconstructive networks. We integrate this framework with a reconstruction-based approach and concurrently propose a split training strategy.
  arXiv  Detail & Related papers  (2023-08-29T07:00:35Z)
• RISP: Rendering-Invariant State Predictor with Differentiable Simulation and Rendering for Cross-Domain Parameter Estimation [110.4255414234771]
  Existing solutions require massive training data or lack generalizability to unknown rendering configurations. We propose a novel approach that marries domain randomization and differentiable rendering gradients to address this problem. Our approach achieves significantly lower reconstruction errors and has better generalizability among unknown rendering configurations.
  arXiv  Detail & Related papers  (2022-05-11T17:59:51Z)
• SCNet: A Generalized Attention-based Model for Crack Fault Segmentation [0.0]
  Anomaly detection and localization is an important vision problem, having multiple applications. Periodic health monitoring and fault (anomaly) detection in vast infrastructures is one such application area of vision-based anomaly segmentation. Cracks are critical and frequent surface faults that manifest as extreme zigzag-shaped thin, elongated regions. In this work, we address an open aspect of automatic crack segmentation problem, that of generalizing and improving the performance of segmentation across a variety of scenarios, by modeling the problem differently.
  arXiv  Detail & Related papers  (2021-12-02T17:16:18Z)
• A Procedural World Generation Framework for Systematic Evaluation of Continual Learning [2.599882743586164]
  We introduce a computer graphics simulation framework that repeatedly renders only upcoming urban scene fragments. At its core lies a modular parametric generative model with adaptable generative factors.
  arXiv  Detail & Related papers  (2021-06-04T16:31:43Z)
• Data-driven generation of plausible tissue geometries for realistic photoacoustic image synthesis [53.65837038435433]
  Photoacoustic tomography (PAT) has the potential to recover morphological and functional tissue properties. We propose a novel approach to PAT data simulation, which we refer to as "learning to simulate" We leverage the concept of Generative Adversarial Networks (GANs) trained on semantically annotated medical imaging data to generate plausible tissue geometries.
  arXiv  Detail & Related papers  (2021-03-29T11:30:18Z)
• Generative Partial Visual-Tactile Fused Object Clustering [81.17645983141773]
  We propose a Generative Partial Visual-Tactile Fused (i.e., GPVTF) framework for object clustering. A conditional cross-modal clustering generative adversarial network is then developed to synthesize one modality conditioning on the other modality. To the end, two pseudo-label based KL-divergence losses are employed to update the corresponding modality-specific encoders.
  arXiv  Detail & Related papers  (2020-12-28T02:37:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.