Benchmarking bandgap prediction in semiconductors under experimental and realistic evaluation settings

• URL: http://arxiv.org/abs/2604.25568v1
• Date: Tue, 28 Apr 2026 12:39:21 GMT
• Title: Benchmarking bandgap prediction in semiconductors under experimental and realistic evaluation settings
• Abstract summary: RealMat-BaG is a benchmark for assessing model reliability under experimentally relevant conditions.<n>We curate an open-access dataset of experimental bandgaps with aligned crystal structures and compare graph neural networks.
• Score: 41.716484717366264
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Accurate bandgap prediction is crucial for semiconductor applications, yet machine learning models trained on computational data often struggle to generalize to experimental bandgap measurements. Challenges related to data fidelity, domain generalization, and model interpretability remain insufficiently addressed in existing evaluation frameworks. To bridge this gap, we introduce RealMat-BaG, a benchmark for assessing model reliability under experimentally relevant conditions. We curate an open-access dataset of experimental bandgaps with aligned crystal structures and compare graph neural networks as well as classical machine learning baselines. Our framework evaluates performance across statistical and domain-based splits, examines transfer from DFT-computed to experimental bandgaps, and analyzes interpretability at both elemental-property and structural levels. Our results reveal the fundamental generalization limitations of current bandgap prediction models and establish a benchmark aligned with experimental measurements for developing more reliable learning strategies for materials discovery.

Related papers

• A comprehensive and trustworthy benchmark of AI methods for change detection in Earth observation [6.608286365425497]
  Change detection in Earth observation is critical for monitoring land surface transformations.<n>We present a standardized, open-source benchmark for evaluating state-of-the-art (SOTA) deep learning methods for change detection.
  arXiv  Detail & Related papers  (2026-08-28T12:04:39Z)
• A Critical Audit of Spatiotemporal Forecasting Benchmark Datasets and Baselines [19.152026148047536]
  We take a step back and analyse the benchmark datasets via classical time series methods to uncover why spatially-unaware linear models pose a stronger competitor than previously reported.<n>Our statistical analysis provides a toolset for identifying significant spatial and temporal correlations.<n>By applying the results of our analysis to a simple hybrid model, we show how our methodology can lead to novel ways of developing GNN models.
  arXiv  Detail & Related papers  (2026-08-21T11:08:17Z)
• Overcoming the Modality Gap in Context-Aided Forecasting [54.976964834365056]
  We introduce a semi-synthetic data augmentation method that generates contexts both descriptive of temporal dynamics and verifiably complementary to numerical histories.<n>This approach enables massive-scale dataset creation, resulting in CAF-7M, a corpus of 7 million context-augmented time series windows.
  arXiv  Detail & Related papers  (2026-03-12T21:05:33Z)
• The Benchmarking Epistemology: Construct Validity for Evaluating Machine Learning Models [1.1315617886931963]
  We develop conditions of construct validity inspired by psychological measurement theory.<n>We examine these assumptions in practice through three case studies.<n>Our framework clarifies conditions under which benchmark scores can support diverse scientific claims.
  arXiv  Detail & Related papers  (2025-10-27T10:30:30Z)
• Evaluating Universal Machine Learning Force Fields Against Experimental Measurements [15.863801293927635]
  Universal machine learning force fields (UMLFFs) promise to revolutionize materials science by enabling rapid atomistic simulations across the periodic table.<n>Here, we present UniFFBench, a comprehensive framework for evaluating experimental bondingFFs against experimental measurements of 1,500 carefully curated mineral structures.<n>Our systematic evaluation of six state-of-the-artFFs reveals a substantial reality gap: models achieving impressive performance on computational benchmarks often fail when confronted with experimental complexity.
  arXiv  Detail & Related papers  (2025-08-07T18:21:39Z)
• Can We Validate Counterfactual Estimations in the Presence of General Network Interference? [13.49152464081862]
  We introduce a framework that facilitates the use of machine learning tools for both estimation and validation in causal inference.<n>New distribution-preserving network bootstrap generates statistically-valid subpopulations from a single experiment's data.<n>Counterfactual cross-validation procedure adapts the principles of model validation to the unique constraints of causal settings.
  arXiv  Detail & Related papers  (2025-02-03T06:51:04Z)
• Testing Generalizability in Causal Inference [3.547529079746247]
  No formal procedure exists for statistically evaluating generalizability in machine learning algorithms.<n>We propose a systematic framework for statistically evaluating the generalizability of high-dimensional causal inference models.
  arXiv  Detail & Related papers  (2024-11-05T11:44:00Z)
• Fast and Reliable Probabilistic Reflectometry Inversion with Prior-Amortized Neural Posterior Estimation [73.81105275628751]
  Finding all structures compatible with reflectometry data is computationally prohibitive for standard algorithms. We address this lack of reliability with a probabilistic deep learning method that identifies all realistic structures in seconds. Our method, Prior-Amortized Neural Posterior Estimation (PANPE), combines simulation-based inference with novel adaptive priors.
  arXiv  Detail & Related papers  (2024-07-26T10:29:16Z)
• GenBench: A Benchmarking Suite for Systematic Evaluation of Genomic Foundation Models [56.63218531256961]
  We introduce GenBench, a benchmarking suite specifically tailored for evaluating the efficacy of Genomic Foundation Models. GenBench offers a modular and expandable framework that encapsulates a variety of state-of-the-art methodologies. We provide a nuanced analysis of the interplay between model architecture and dataset characteristics on task-specific performance.
  arXiv  Detail & Related papers  (2024-06-01T08:01:05Z)
• CausalBench: A Large-scale Benchmark for Network Inference from Single-cell Perturbation Data [61.088705993848606]
  We introduce CausalBench, a benchmark suite for evaluating causal inference methods on real-world interventional data. CaulBench incorporates biologically-motivated performance metrics, including new distribution-based interventional metrics.
  arXiv  Detail & Related papers  (2022-10-31T13:04:07Z)
• Estimating the Effects of Continuous-valued Interventions using Generative Adversarial Networks [103.14809802212535]
  We build on the generative adversarial networks (GANs) framework to address the problem of estimating the effect of continuous-valued interventions. Our model, SCIGAN, is flexible and capable of simultaneously estimating counterfactual outcomes for several different continuous interventions. To address the challenges presented by shifting to continuous interventions, we propose a novel architecture for our discriminator.
  arXiv  Detail & Related papers  (2020-02-27T18:46:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.