Related papers: Births are difficult to predict even with rich survey and full-population register data

Births are difficult to predict even with rich survey and full-population register data

URL: http://arxiv.org/abs/2609.01194v1
Date: Tue, 01 Sep 2026 13:04:29 GMT
Title: Births are difficult to predict even with rich survey and full-population register data
Abstract summary: We examine one outcome - having a child within three years - through a near-ideal setting for prediction.<n>147 researchers predicted births for Dutch residents aged 18-45 using survey data and full-population registers.
Score: 35.78566697533737
License: http://creativecommons.org/licenses/by/4.0/
Abstract: Major life events have proven difficult to predict. Does this reflect limits of theory, data, and algorithms, or the large role of chance? We examine one outcome - having a child within three years - through a near-ideal setting for prediction: a data challenge where 147 researchers predicted births for Dutch residents aged 18-45, using survey data and full-population registers. Methods ranged from logistic regression to a large language model and transformers. Predictions were moderately accurate (best F1: register 0.59, survey 0.76); advanced models did not outperform classical ones; and the larger registers did not beat the survey. Simulating the stochastic biology of conception and pregnancy, we estimated a predictive ceiling (survey F1 ~ 0.86-0.94, register 0.88-0.96). Observed performance falls short of this ceiling, implicating imperfect data, methods, and unmodelled chance, while the ceiling itself shows that chance in reproduction alone sets a non-trivial limit on predicting individual lives.

Related papers

Predicting gestational age at birth in the context of preterm birth from multi-modal fetal MRI [0.7150253772413703]
Preterm birth is associated with significant mortality and a risk for lifelong morbidity.<n>A pipeline consisting of bespoke machine learning methods was developed to predict gestational age (GA) at birth.<n>The pipeline achieves an R2 score of 0.13 and a mean absolute error of 2.74 weeks.
arXiv Detail & Related papers (2026-06-18T12:39:40Z)
Bayesian Elicitation with LLMs: Model Size Helps, Extra "Reasoning" Doesn't Always [0.0]
Large language models (LLMs) have been proposed as alternatives to human experts for estimating unknown quantities with associated uncertainty.<n>We test this by asking eleven LLMs to estimate population statistics, such as health prevalence rates, personality trait distributions, and labor market figures.<n>We vary each model's reasoning effort to test whether more "thinking" improves results.
arXiv Detail & Related papers (2026-04-02T11:00:13Z)
Epistemic Reject Option Prediction [6.531546527140475]
In high-stakes applications, predictive models must not only produce accurate predictions but also quantify and communicate their uncertainty.<n>Reject-option prediction addresses this by allowing the model to abstain when prediction uncertainty is high.<n>This is the first principled framework that enables learning predictors capable of identifying inputs for which the training data is insufficient to make reliable decisions.
arXiv Detail & Related papers (2025-11-06T22:39:21Z)
A deep learning and machine learning approach to predict neonatal death in the context of São Paulo [0.0]
Neonatal death is still a concerning reality for underdeveloped and even some developed countries.<n>To reduce this number, early prediction of endangered babies is crucial.<n>Machine learning was used to determine whether a newborn baby is at risk.
arXiv Detail & Related papers (2025-06-20T11:44:48Z)
DataDecide: How to Predict Best Pretraining Data with Small Experiments [67.95896457895404]
We release models, data, and evaluations in DataDecide -- the most extensive open suite of models over differences in data and scale.<n>We conduct controlled pretraining experiments across 25 corpora with differing sources, deduplication, and filtering up to 100B tokens, model sizes up to 1B parameters, and 3 random seeds.
arXiv Detail & Related papers (2025-04-15T17:02:15Z)
Prediction-Powered Causal Inferences [59.98498488132307]
We focus on Prediction-Powered Causal Inferences (PPCI)<n>We first show that conditional calibration guarantees valid PPCI at population level.<n>We then introduce a sufficient representation constraint transferring validity across experiments.
arXiv Detail & Related papers (2025-02-10T10:52:17Z)
Understanding overfitting in random forest for probability estimation: a visualization and simulation study [1.7103406524471811]
In a case study on predicting ovarian malignancy, we observed training c-statistics close to 1. Although this suggests overfitting, performance was competitive on test data. We aimed to understand the behaviour of random forests by visualizing data space in three real world case studies and a simulation study.
arXiv Detail & Related papers (2024-02-28T15:29:02Z)
Quantifying predictive uncertainty of aphasia severity in stroke patients with sparse heteroscedastic Bayesian high-dimensional regression [47.1405366895538]
Sparse linear regression methods for high-dimensional data commonly assume that residuals have constant variance, which can be violated in practice. This paper proposes estimating high-dimensional heteroscedastic linear regression models using a heteroscedastic partitioned empirical Bayes Expectation Conditional Maximization algorithm.
arXiv Detail & Related papers (2023-09-15T22:06:29Z)
Uncertainty estimation of pedestrian future trajectory using Bayesian approximation [137.00426219455116]
Under dynamic traffic scenarios, planning based on deterministic predictions is not trustworthy. The authors propose to quantify uncertainty during forecasting using approximation which deterministic approaches fail to capture. The effect of dropout weights and long-term prediction on future state uncertainty has been studied.
arXiv Detail & Related papers (2022-05-04T04:23:38Z)
Conformal Prediction Under Feedback Covariate Shift for Biomolecular Design [56.86533144730384]
We introduce a method to quantify predictive uncertainty in settings where the training and test data are statistically dependent.<n>As a motivating use case, we demonstrate with several real data sets how our method quantifies uncertainty for the predicted fitness of designed proteins.
arXiv Detail & Related papers (2022-02-08T02:59:12Z)
Improving Uncertainty Calibration via Prior Augmented Data [56.88185136509654]
Neural networks have proven successful at learning from complex data distributions by acting as universal function approximators. They are often overconfident in their predictions, which leads to inaccurate and miscalibrated probabilistic predictions. We propose a solution by seeking out regions of feature space where the model is unjustifiably overconfident, and conditionally raising the entropy of those predictions towards that of the prior distribution of the labels.
arXiv Detail & Related papers (2021-02-22T07:02:37Z)
A random shuffle method to expand a narrow dataset and overcome the associated challenges in a clinical study: a heart failure cohort example [50.591267188664666]
The aim of this study was to design a random shuffle method to enhance the cardinality of an HF dataset while it is statistically legitimate. The proposed random shuffle method was able to enhance the HF dataset cardinality circa 10 times and circa 21 times when followed by a random repeated-measures approach.
arXiv Detail & Related papers (2020-12-12T10:59:38Z)
Ambiguity in Sequential Data: Predicting Uncertain Futures with Recurrent Models [110.82452096672182]
We propose an extension of the Multiple Hypothesis Prediction (MHP) model to handle ambiguous predictions with sequential data. We also introduce a novel metric for ambiguous problems, which is better suited to account for uncertainties.
arXiv Detail & Related papers (2020-03-10T09:15:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.