TabPFN Extensions for Interpretable Geotechnical Modelling

• URL: http://arxiv.org/abs/2603.21033v1
• Date: Sun, 22 Mar 2026 03:06:28 GMT
• Title: TabPFN Extensions for Interpretable Geotechnical Modelling
• Abstract summary: This paper presents an investigation into the use of TabPFN, a transformer-based foundation model using in-context learning.<n>We apply cosine-similarity analysis to TabPFN-derived embeddings, visualise full posterior distributions from an iterative inference procedure, and compute SHAP-based feature importance.<n>Results suggest the potential of foundation-model-based tools to support interpretable, uncertainty-aware parameter inference in data-scarce geotechnical practice.
• Score: 1.89576312978177
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Geotechnical site characterisation relies on sparse, heterogeneous borehole data where uncertainty quantification and model interpretability are as critical as predictive accuracy for reliable engineering decisions. This paper presents an exploratory investigation into the use of TabPFN, a transformer-based tabular foundation model using in-context learning, and its extension library tabpfn-extensions for two geotechnical inference tasks: (1) soil-type classification using N-value and shear-wave velocity data from a synthetic geotechnical dataset, and (2) iterative imputation of five missing mechanical parameters ($s_\mathrm{u}$, $E_{\mathrm{u}}$, ${σ'}_\mathrm{p}$, $C_\mathrm{c}$, $C_\mathrm{v}$) in benchmark problem BM/AirportSoilProperties/2/2025. We apply cosine-similarity analysis to TabPFN-derived embeddings, visualise full posterior distributions from an iterative inference procedure, and compute SHAP-based feature importance, all without model retraining. Learned embeddings clearly separate Clay and Sand samples without explicit soil-type supervision; iterative imputation improves predictions for four of five target parameters, with posterior widths that reflect physically reasonable parameter-specific uncertainty; and SHAP analysis reveals the inter-parameter dependency structure, recovering established geotechnical relationships including the Skempton compression index correlation and the inverse dependence of preconsolidation pressure on water content. These results suggest the potential of foundation-model-based tools to support interpretable, uncertainty-aware parameter inference in data-scarce geotechnical practice.

Related papers

• Missing Data Imputation under Manifold Hypothesis [0.0]
  manifold hypothesis posits that high-dimensional data are concentrated near a low-dimensional embedded manifold.<n>Recent advances in mixture variational autoencoders (VAEs) provide a powerful tool for extracting such underlying structure in a faithful manner.<n>We propose a model-based imputation method that enables sampling from ( p(bmx_mathrmmis mid bmx_mathrmobs) ) via a sampling-importance-resampling (SIR) procedure.
  arXiv  Detail & Related papers  (2026-07-03T23:42:33Z)
• Joint Model and Data Sparsification via the Marginal Likelihood [53.29070892356214]
  We propose jointly learning individual feature and sample relevancies, enabling simultaneous model and data sparsification.<n>This symmetric pruning of model and data offers a natural extension that preserves conjugacy.<n> Empirical results across diverse regression tasks affirm that a joint ARD approach consistently yields both sparse and robust prediction models.
  arXiv  Detail & Related papers  (2026-05-28T13:26:53Z)
• A Mean Curvature Approach to Boundary Detection: Geometric Insights for Unsupervised Learning [52.452902154360565]
  We introduce Mean Curvature Boundary Points (MCBP), a novel geometric framework grounded in Geometric Machine Learning.<n>MCBP explicitly models the intrinsic curvature of the data manifold and computes pointwise mean curvature without requiring principled manifold parametrization.<n>Experiments on synthetic and real-world datasets demonstrate that MCBP consistently improves clustering performance.
  arXiv  Detail & Related papers  (2026-05-05T20:19:09Z)
• TopoPrune: Robust Data Pruning via Unified Latent Space Topology [9.204659134755792]
  We introduce TopoPrune, a framework which resolves this challenge by leveraging topology to capture the stable, intrinsic structure of data.<n>TopoPrune operates at two scales, (1) utilizing a topology-aware manifold approximation to establish a global low-dimensional embedding of the dataset.<n>We demonstrate that our unified dual-scale topological approach ensures high accuracy and precision, particularly at significant dataset pruning rates.
  arXiv  Detail & Related papers  (2026-02-02T19:53:59Z)
• Leveraging generative adversarial networks with spatially adaptive denormalization for multivariate stochastic seismic data inversion [0.0]
  We propose an iterative geostatistical inversion algorithm, SPADE-GANInv, for the prediction of facies and multiple correlated continuous properties from seismic data.<n>The SPADE-GAN is trained to reproduce realistic geometries, while sequential co-simulation predicts the spatial variability of the facies-dependent continuous properties.<n>The method is demonstrated on both 2-D synthetic scenarios and field data, targeting the prediction of facies, porosity, and acoustic impedance from full-stack seismic data.
  arXiv  Detail & Related papers  (2025-12-02T15:25:22Z)
• Nonparametric Data Attribution for Diffusion Models [57.820618036556084]
  Data attribution for generative models seeks to quantify the influence of individual training examples on model outputs.<n>We propose a nonparametric attribution method that operates entirely on data, measuring influence via patch-level similarity between generated and training images.
  arXiv  Detail & Related papers  (2025-10-16T03:37:16Z)
• Tabular foundation model for GEOAI benchmark problems BM/AirportSoilProperties/2/2025 [2.07098502859192]
  This paper presents a novel application of the Tabular Prior-Data Fitted Network (TabPFN) to site characterization problems defined in the GEOAI benchmark BM/AirportSoilProperties/2/2025.<n>We apply TabPFN in a zero-training, few-shot, in-spatial learning setting and provide it with additional context from the big indirect database (BID)<n>The study demonstrates that TabPFN, as a general-purpose foundation model, achieved superior accuracy and well-calibrated predictive distributions.
  arXiv  Detail & Related papers  (2025-09-03T10:21:18Z)
• CP$^2$: Leveraging Geometry for Conformal Prediction via Canonicalization [51.716834831684004]
  We study the problem of conformal prediction (CP) under geometric data shifts.<n>We propose integrating geometric information--such as geometric pose--into the conformal procedure to reinstate its guarantees.
  arXiv  Detail & Related papers  (2025-06-19T10:12:02Z)
• DispFormer: A Pretrained Transformer Incorporating Physical Constraints for Dispersion Curve Inversion [56.64622091009756]
  This study introduces DispFormer, a transformer-based neural network for $v_s$ profile inversion from Rayleigh-wave phase and group dispersion curves.<n>DispFormer processes dispersion data independently at each period, allowing it to handle varying lengths without requiring network modifications or strict alignment between training and testing datasets.
  arXiv  Detail & Related papers  (2025-01-08T09:08:24Z)
• Latent Semantic Consensus For Deterministic Geometric Model Fitting [109.44565542031384]
  We propose an effective method called Latent Semantic Consensus (LSC) LSC formulates the model fitting problem into two latent semantic spaces based on data points and model hypotheses. LSC is able to provide consistent and reliable solutions within only a few milliseconds for general multi-structural model fitting.
  arXiv  Detail & Related papers  (2024-03-11T05:35:38Z)
• Conditional Korhunen-Lo\'{e}ve regression model with Basis Adaptation for high-dimensional problems: uncertainty quantification and inverse modeling [62.997667081978825]
  We propose a methodology for improving the accuracy of surrogate models of the observable response of physical systems. We apply the proposed methodology to constructing surrogate models via the Basis Adaptation (BA) method of the stationary hydraulic head response.
  arXiv  Detail & Related papers  (2023-07-05T18:14:38Z)
• Topological Parallax: A Geometric Specification for Deep Perception Models [0.778001492222129]
  We introduce topological parallax as a theoretical and computational tool that compares a trained model to a reference dataset. Our examples show that this geometric similarity between dataset and model is essential to trustworthy and perturbation. This new concept will add value to the current debate regarding the unclear relationship between overfitting and generalization in applications of deep-learning.
  arXiv  Detail & Related papers  (2023-06-20T18:45:24Z)
• Semiparametric Bayesian Forecasting of Spatial Earthquake Occurrences [77.68028443709338]
  We propose a fully Bayesian formulation of the Epidemic Type Aftershock Sequence (ETAS) model. The occurrence of the mainshock earthquakes in a geographical region is assumed to follow an inhomogeneous spatial point process.
  arXiv  Detail & Related papers  (2020-02-05T10:11:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.