Position: AI for Science Should Treat Measurement-to-Dataset Pipelines as Inference Components

• URL: http://arxiv.org/abs/2605.24558v1
• Date: Sat, 23 May 2026 12:50:00 GMT
• Title: Position: AI for Science Should Treat Measurement-to-Dataset Pipelines as Inference Components
• Abstract summary: In domains relying on emphindirect observation, the learner observes a derivative representation produced by multi-stage measurement, reconstruction, and preprocessing pipelines.<n>We identify three failure modes arising from this frozen lens''<n>We call on the AI4Science community to make pipelines emphcomputable inference objects via domain-specific Computable Observation Frameworks.
• Score: 2.439525434311725
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: AI for Science (AI4Science) workflows often treat the released dataset as a fixed interface to the underlying system. However, in domains relying on \emph{indirect observation}, the learner observes a derivative representation produced by multi-stage measurement, reconstruction, and preprocessing pipelines. \textbf{We argue that these measurement-to-dataset pipelines are inference components: treating their outputs as ``given data'' freezes an observation model and obscures uncertainty over feasible pipeline choices.} We identify three failure modes arising from this ``frozen lens'': \textbf{(C1) hidden hypothesis space}, where the released dataset does not specify the pipeline configuration or its validity conditions; \textbf{(C2) uncertified transportability}, where a pipeline may be documented but its regime of validity is untested, so failures under distribution shift cannot be adjudicated; \textbf{(C3) ungoverned multiplicity}, where many defensible pipelines exist and dispersion is real but not propagated into uncertainty-aware evidence. We stress-test these claims with a large-scale neuroscience empirical audit, finding a survival rate of $\approx 0.0004\%$ under a cross-dataset stability criterion. We call on the AI4Science community to make pipelines \emph{computable} inference objects via domain-specific Computable Observation Frameworks. This shift enables quantifying pipeline adequacy and stability, converting implicit implementation choices into auditable, reproducible, and cumulative scientific evidence.

Related papers

• Is Convergence Inevitable? Tracing Output Homogeneity Back to Base Models [49.46165710676499]
  We find that semantic convergence is observed from the first alignment stage.<n>We find that convergence can be revealed and amplified, but not introduced by the SFT data.<n> Taken together, our results suggest that semantic convergence may arise naturally from the objectives underlying LM training.
  arXiv  Detail & Related papers  (2026-08-11T20:47:06Z)
• Certifying Plans under Model Mismatch: A Trilemma for Reachability from Scarce Data [5.455772242709494]
  Sim-to-real policies are designed under nominal dynamics, but target-system trials may yield only a few isolated one-step transitions.<n>We study pre-execution certification of a fixed control sequence, such as an action chunk produced by a learned policy.
  arXiv  Detail & Related papers  (2026-08-03T16:28:26Z)
• Conformal Changepoint Localization and Root Cause Analysis with Corrupted Observations [55.76952683833966]
  This paper proposes weighted CONCH (W-CONCH) and weighted CROC (W-CROC) to reduce confidence set size when data may be corrupted.<n>Experiments on image-based and real-world changepoint and root-cause benchmarks show that uncertainty-based weighting substantially reduces confidence set size.
  arXiv  Detail & Related papers  (2026-07-29T05:16:59Z)
• What Accuracy and Gradient Cosine Miss: Evaluating Feedback Alignment via Scale Stability, Reference Validity, and Depth Utility [48.10132234701036]
  We propose a diagnostic evaluation protocol based on three checks -- scale stability, reference validity, and depth utility -- together with per-layer rather than aggregate cosine reporting.<n>Across multiple architectures and methods, our protocol identifies all failures with wide calibration margins.
  arXiv  Detail & Related papers  (2026-06-19T06:04:17Z)
• Trust, but Don't Verify: Epistemic Blind Spots in LLM Source Evaluation [0.0]
  We show that models possess the capability to detect fabricated statistics but do not recruit this capability during multi-source synthesis.<n>Specifically, source influence is governed by a methodology-register gate that responds to the distributional register of analytical text but not to numeric validity.
  arXiv  Detail & Related papers  (2026-06-03T20:15:48Z)
• Non-Parametric Rehearsal Learning via Conditional Mean Embeddings [88.89267783967263]
  We propose a non-parametric rehearsal learning approach for textitavoiding undesired future (AUF) problems.<n>Specifically, we use kernel machinery to reformulate the AUF objective into a unified representation that disentangles desirability modeling from action-induced distributional changes.
  arXiv  Detail & Related papers  (2026-05-09T15:30:52Z)
• Data-Driven Reachability Analysis via Diffusion Models with PAC Guarantees [3.651951296125688]
  We present a data-driven framework for reachability analysis of nonlinear dynamical systems.<n>A denoising diffusion probabilistic model learns the time-evolving state distribution of a dynamical system from trajectory data alone.
  arXiv  Detail & Related papers  (2026-03-31T22:14:36Z)
• Robust Sequential Tracking via Bounded Information Geometry and Non-Parametric Field Actions [0.0]
  Empirical benchmarks across three domains--LiDAR maneuvering target tracking, high-frequency cryptocurrency order flow, and quantum state tomography--demonstrate that this information geometry analytically truncates outliers.
  arXiv  Detail & Related papers  (2026-03-13T21:40:44Z)
• Causal Identification from Counterfactual Data: Completeness and Bounding Results [54.147490305295456]
  We develop an algorithm for identifying counterfactual queries from an arbitrary set of Layer 3 distributions.<n>We establish the theoretical limit of which counterfactuals can be identified from physically realizable distributions.<n>We derive novel analytic bounds for such quantities using realizable counterfactual data.
  arXiv  Detail & Related papers  (2026-02-26T22:48:12Z)
• The Inevitability of Side-Channel Leakage in Encrypted Traffic [18.365496056967917]
  We show that in efficiency-prioritized systems, leakage is inevitable when at least one application pair is distinguishable.<n>This establishes the first rigorous information-theoretic foundation for encrypted traffic side channels.
  arXiv  Detail & Related papers  (2026-02-15T08:55:19Z)
• Stable but Wrong: When More Data Degrades Scientific Conclusions [26.934158434915133]
  We show that modern science increasingly relies on ever-growing observational datasets and automated inference pipelines.<n>We identify a structural regime in which standard inference procedures converge smoothly, remain well calibrated, and pass conventional diagnostic checks.<n>This failure arises when the reliability of observations degrades in a manner that is intrinsically unobservable to the inference process itself.
  arXiv  Detail & Related papers  (2026-02-05T13:51:47Z)
• A Sample Efficient Conditional Independence Test in the Presence of Discretization [54.047334792855345]
  Conditional Independence (CI) tests directly to discretized data can lead to incorrect conclusions.<n>Recent advancements have sought to infer the correct CI relationship between the latent variables through binarizing observed data.<n>Motivated by this, this paper introduces a sample-efficient CI test that does not rely on the binarization process.
  arXiv  Detail & Related papers  (2025-06-10T12:41:26Z)
• Error-quantified Conformal Inference for Time Series [55.11926160774831]
  Uncertainty quantification in time series prediction is challenging due to the temporal dependence and distribution shift on sequential data.<n>We propose itError-quantified Conformal Inference (ECI) by smoothing the quantile loss function.<n>ECI can achieve valid miscoverage control and output tighter prediction sets than other baselines.
  arXiv  Detail & Related papers  (2025-02-02T15:02:36Z)
• Data-Driven Reachability Analysis of Stochastic Dynamical Systems with Conformal Inference [1.446438366123305]
  We consider data-driven reachability analysis of discrete-time dynamical systems using conformal inference. We focus on learning-enabled control systems with complex closed-loop dynamics.
  arXiv  Detail & Related papers  (2023-09-17T07:23:01Z)
• From Fake to Real: Pretraining on Balanced Synthetic Images to Prevent Spurious Correlations in Image Recognition [64.59093444558549]
  We propose a simple, easy-to-implement, two-step training pipeline that we call From Fake to Real. By training on real and synthetic data separately, FFR does not expose the model to the statistical differences between real and synthetic data. Our experiments show that FFR improves worst group accuracy over the state-of-the-art by up to 20% over three datasets.
  arXiv  Detail & Related papers  (2023-08-08T19:52:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.