Beyond development: Challenges in deploying machine learning models for structural engineering applications

• URL: http://arxiv.org/abs/2404.12544v1
• Date: Thu, 18 Apr 2024 23:40:42 GMT
• Title: Beyond development: Challenges in deploying machine learning models for structural engineering applications
• Authors: Mohsen Zaker Esteghamati, Brennan Bean, Henry V. Burton, M. Z. Naser,
• Abstract summary: This paper aims to illustrate the challenges of developing machine learning models suitable for deployment through two illustrative examples. Among various pitfalls, the presented discussion focuses on model overfitting and underspecification, training data representativeness, variable omission bias, and cross-validation. Results highlight the importance of implementing rigorous model validation techniques through adaptive sampling, careful physics-informed feature selection, and considerations of both model complexity and generalizability.
• Score: 2.6415688445750383
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine learning (ML)-based solutions are rapidly changing the landscape of many fields, including structural engineering. Despite their promising performance, these approaches are usually only demonstrated as proof-of-concept in structural engineering, and are rarely deployed for real-world applications. This paper aims to illustrate the challenges of developing ML models suitable for deployment through two illustrative examples. Among various pitfalls, the presented discussion focuses on model overfitting and underspecification, training data representativeness, variable omission bias, and cross-validation. The results highlight the importance of implementing rigorous model validation techniques through adaptive sampling, careful physics-informed feature selection, and considerations of both model complexity and generalizability.

Related papers

• Structuring a Training Strategy to Robustify Perception Models with Realistic Image Augmentations [1.5723316845301678]
  This report introduces a novel methodology for training with augmentations to enhance model robustness and performance in such conditions. We present a comprehensive framework that includes identifying weak spots in Machine Learning models, selecting suitable augmentations, and devising effective training strategies. Experimental results demonstrate improvements in model performance, as measured by commonly used metrics such as mean Average Precision (mAP) and mean Intersection over Union (mIoU) on open-source object detection and semantic segmentation models and datasets.
  arXiv  Detail & Related papers  (2024-08-30T14:15:48Z)
• Theoretical Insights into Overparameterized Models in Multi-Task and Replay-Based Continual Learning [37.745896674964186]
  Multi-task learning (MTL) aims to improve the generalization performance of a model on multiple related tasks by training it simultaneously on those tasks. Continual learning (CL) involves adapting to new sequentially arriving tasks over time without forgetting the previously acquired knowledge. We develop theoretical results describing the effect of various system parameters on the model's performance in an MTL setup. Our results reveal the impact of buffer size and model capacity on the forgetting rate in a CL setup and help shed light on some of the state-of-the-art CL methods.
  arXiv  Detail & Related papers  (2024-08-29T23:22:40Z)
• Mining Frequent Structures in Conceptual Models [2.841785306638839]
  We propose a general approach to the problem of discovering frequent structures in conceptual modeling languages. We use the combination of a frequent subgraph mining algorithm and graph manipulation techniques. The primary objective is to offer a support facility for language engineers.
  arXiv  Detail & Related papers  (2024-06-11T10:24:02Z)
• What matters when building vision-language models? [52.8539131958858]
  We develop Idefics2, an efficient foundational vision-language model with 8 billion parameters. Idefics2 achieves state-of-the-art performance within its size category across various multimodal benchmarks. We release the model (base, instructed, and chat) along with the datasets created for its training.
  arXiv  Detail & Related papers  (2024-05-03T17:00:00Z)
• OtterHD: A High-Resolution Multi-modality Model [57.16481886807386]
  OtterHD-8B is an innovative multimodal model engineered to interpret high-resolution visual inputs with granular precision. Our study highlights the critical role of flexibility and high-resolution input capabilities in large multimodal models.
  arXiv  Detail & Related papers  (2023-11-07T18:59:58Z)
• ZhiJian: A Unifying and Rapidly Deployable Toolbox for Pre-trained Model Reuse [59.500060790983994]
  This paper introduces ZhiJian, a comprehensive and user-friendly toolbox for model reuse, utilizing the PyTorch backend. ZhiJian presents a novel paradigm that unifies diverse perspectives on model reuse, encompassing target architecture construction with PTM, tuning target model with PTM, and PTM-based inference.
  arXiv  Detail & Related papers  (2023-08-17T19:12:13Z)
• Aligning Optimization Trajectories with Diffusion Models for Constrained Design Generation [17.164961143132473]
  We introduce a learning framework that demonstrates the efficacy of aligning the sampling trajectory of diffusion models with the optimization trajectory derived from traditional physics-based methods. Our method allows for generating feasible and high-performance designs in as few as two steps without the need for expensive preprocessing, external surrogate models, or additional labeled data. Our results demonstrate that TA outperforms state-of-the-art deep generative models on in-distribution configurations and halves the inference computational cost.
  arXiv  Detail & Related papers  (2023-05-29T09:16:07Z)
• Scaling Vision-Language Models with Sparse Mixture of Experts [128.0882767889029]
  We show that mixture-of-experts (MoE) techniques can achieve state-of-the-art performance on a range of benchmarks over dense models of equivalent computational cost. Our research offers valuable insights into stabilizing the training of MoE models, understanding the impact of MoE on model interpretability, and balancing the trade-offs between compute performance when scaling vision-language models.
  arXiv  Detail & Related papers  (2023-03-13T16:00:31Z)
• Generalization Properties of Retrieval-based Models [50.35325326050263]
  Retrieval-based machine learning methods have enjoyed success on a wide range of problems. Despite growing literature showcasing the promise of these models, the theoretical underpinning for such models remains underexplored. We present a formal treatment of retrieval-based models to characterize their generalization ability.
  arXiv  Detail & Related papers  (2022-10-06T00:33:01Z)
• Beyond Explaining: Opportunities and Challenges of XAI-Based Model Improvement [75.00655434905417]
  Explainable Artificial Intelligence (XAI) is an emerging research field bringing transparency to highly complex machine learning (ML) models. This paper offers a comprehensive overview over techniques that apply XAI practically for improving various properties of ML models. We show empirically through experiments on toy and realistic settings how explanations can help improve properties such as model generalization ability or reasoning.
  arXiv  Detail & Related papers  (2022-03-15T15:44:28Z)
• Problem-fluent models for complex decision-making in autonomous materials research [0.0]
  We highlight the coupling of machine learning methods and models and more problem-aware modeling. We review the general Bayesian framework for closed-loop design employed by many autonomous materials platforms.
  arXiv  Detail & Related papers  (2021-03-13T19:23:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.