From field-scale to large-scale spectral libraries: Tabular foundation models in soil spectroscopy

• URL: http://arxiv.org/abs/2608.00608v1
• Date: Sat, 01 Aug 2026 11:56:50 GMT
• Title: From field-scale to large-scale spectral libraries: Tabular foundation models in soil spectroscopy
• Abstract summary: Visible and near-infrared (vis-NIR) and mid-infrared (MIR) spectroscopy enable rapid, cost-effective prediction of soil properties.<n> translating high-dimensional, highly collinear spectra into accurate soil property predictions remains challenging.<n>We investigated regression models and dimensionality reduction approaches for spectroscopic modeling across 85 regression tasks.
• Score: 0.6377541605829954
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Visible and near-infrared (vis-NIR) and mid-infrared (MIR) spectroscopy enable rapid, cost-effective prediction of soil properties. Yet, translating high-dimensional, highly collinear spectra into accurate soil property predictions remains challenging, particularly when employing machine learning. We systematically investigated regression models and dimensionality reduction approaches for spectroscopic modeling across 85 regression tasks from open benchmark datasets in pedometrics spanning field-scale digital soil mapping and a global soil spectral library. We compared an in-context learning tabular foundation model (TabPFN), a convolutional neural network (CNN), rule-based regression (Cubist), Random Forest, and partial least squares regression (PLSR) using full spectra as well as features derived from principal component analysis (PCA) and partial least squares (PLS) latent variables. TabPFN consistently delivered the best overall performance across scales, including large spectral library tasks with tens of thousands of soil samples. Notably, TabPFN applied directly to full spectra already surpassed all classical baselines, showing that explicit dimensionality reduction is not strictly required for strong performance. Further improvements were achieved through PLS, which proved to be an effective dimensionality reduction strategy for all models. Combining PLS latent variables with TabPFN yielded the best predictions overall. Our findings provide evidence-based guidance for spectroscopic calibration model selection across operational scales, demonstrating that the long-standing advantages of PLSR and modern tabular foundation models complement each other in chemometrics.

Related papers

• Tabular foundation models for robust calibration of near-infrared chemical sensing data [0.0]
  Near-infrared spectroscopy is increasingly used as a rapid, non-destructive chemical sensing technology.<n>Tablet foundation models can provide a new calibration strategy for NIR chemical sensing.<n>We benchmark TabPFN on 66 NIR datasets covering 54 regression and 12 classification tasks.
  arXiv  Detail & Related papers  (2026-05-20T09:48:18Z)
• Don't Fix the Basis -- Learn It: Spectral Representation with Adaptive Basis Learning for PDEs [0.2581652423269287]
  We propose Adaptive Basis Learning (ABLE), a framework that learns data-dependent spectral representations instead of relying on predefined bases.<n>ABLE constructs a spatially adaptive Parseval frame via a learned ancillary density, enabling the operator to act in a lifted spectral space.<n>Our results highlight that the data-driven choice of representation, rather than operator complexity alone, is a key bottleneck in neural operator design.
  arXiv  Detail & Related papers  (2026-05-11T12:20:57Z)
• Structure-Informed Estimation for Pilot-Limited MIMO Channels via Tensor Decomposition [51.56484100374058]
  This paper formulates pilot-limited channel estimation as low-rank tensor completion from sparse observations.<n>Experiments on synthetic channels demonstrate 10-20,dB normalized mean-square error (NMSE) improvement over least-squares (LS)<n> evaluations on DeepMIMO ray-tracing channels show 24-44% additional NMSE reduction over pure tensor-based methods.
  arXiv  Detail & Related papers  (2026-02-03T23:38:05Z)
• SIGMA: Scalable Spectral Insights for LLM Collapse [51.863164847253366]
  We introduce SIGMA (Spectral Inequalities for Gram Matrix Analysis), a unified framework for model collapse.<n>By utilizing benchmarks that deriving and deterministic bounds on the matrix's spectrum, SIGMA provides a mathematically grounded metric to track the contraction of the representation space.<n>We demonstrate that SIGMA effectively captures the transition towards states, offering both theoretical insights into the mechanics of collapse.
  arXiv  Detail & Related papers  (2026-01-06T19:47:11Z)
• Self-supervised and Multi-fidelity Learning for Extended Predictive Soil Spectroscopy [2.8830677829565894]
  We propose a framework for multi-fidelity learning and extended predictive soil spectroscopy based on latent space embeddings.<n>A self-supervised representation was pretrained with the large MIR spectral library and the Vari Autoencoder algorithm.<n> Predictions from the spectrum conversion (NIR to MIR) task did not match the performance of the original MIR spectra but were similar or superior to predictive performance of NIR-only models.
  arXiv  Detail & Related papers  (2025-11-20T01:36:33Z)
• Near-Infrared Hyperspectral Imaging Applications in Food Analysis -- Improving Algorithms and Methodologies [0.0]
  This thesis investigates the application of near-infrared hyperspectral imaging (NIR-HSI) for food quality analysis.<n>For several analyses, studies compare models based on convolutional neural networks (CNNs) and partial least squares (PLS)
  arXiv  Detail & Related papers  (2025-10-15T11:53:01Z)
• DispFormer: A Pretrained Transformer Incorporating Physical Constraints for Dispersion Curve Inversion [56.64622091009756]
  This study introduces DispFormer, a transformer-based neural network for $v_s$ profile inversion from Rayleigh-wave phase and group dispersion curves.<n>DispFormer processes dispersion data independently at each period, allowing it to handle varying lengths without requiring network modifications or strict alignment between training and testing datasets.
  arXiv  Detail & Related papers  (2025-01-08T09:08:24Z)
• SpectralEarth: Training Hyperspectral Foundation Models at Scale [47.93167977587301]
  We introduce SpectralEarth, a large-scale multitemporal dataset designed to pretrain hyperspectral foundation models.<n>We pretrain a series of foundation models on SpectralEarth, integrating a spectral adapter into classical vision backbones.<n>In tandem, we construct nine downstream datasets for land-cover, crop-type mapping, and tree-species classification.
  arXiv  Detail & Related papers  (2024-08-15T22:55:59Z)
• Rapid detection of soil carbonates by means of NIR spectroscopy, deep learning methods and phase quantification by powder Xray diffraction [0.0]
  We propose a rapid and efficient way to predict carbonates content in soil by means of FT NIR spectroscopy and by use of deep learning methods. We exploited multiple machine learning methods, such as: 1) a Regressor and 2) a CNN and compare their performance with other traditional ML algorithms.
  arXiv  Detail & Related papers  (2023-07-23T14:32:07Z)
• Boosting Differentiable Causal Discovery via Adaptive Sample Reweighting [62.23057729112182]
  Differentiable score-based causal discovery methods learn a directed acyclic graph from observational data. We propose a model-agnostic framework to boost causal discovery performance by dynamically learning the adaptive weights for the Reweighted Score function, ReScore.
  arXiv  Detail & Related papers  (2023-03-06T14:49:59Z)
• Learning representations with end-to-end models for improved remaining useful life prognostics [64.80885001058572]
  The remaining Useful Life (RUL) of equipment is defined as the duration between the current time and its failure. We propose an end-to-end deep learning model based on multi-layer perceptron and long short-term memory layers (LSTM) to predict the RUL. We will discuss how the proposed end-to-end model is able to achieve such good results and compare it to other deep learning and state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-04-11T16:45:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.