InfoAtlas: A Foundation Model for Zero-Shot Statistical Dependence Estimate

• URL: http://arxiv.org/abs/2606.00241v1
• Date: Fri, 29 May 2026 18:16:51 GMT
• Title: InfoAtlas: A Foundation Model for Zero-Shot Statistical Dependence Estimate
• Abstract summary: InfoAtlas is a foundation model-like architecture that eliminates the need for iterative optimization.<n>It learns to identify diverse dependence structures and predict MI directly from the dataset.<n>Experiments demonstrate that InfoAtlas matches state-of-the-art neural estimators in accuracy while achieving $100times$ speedup.
• Score: 68.14572758582081
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Measuring statistical dependency between high-dimensional random variables is a fundamental task in data science and machine learning. Neural mutual information (MI) estimators offer a promising avenue, but they typically require costly iterative optimization for each new dataset, making them impractical for real-time applications. We present InfoAtlas, a foundation model-like architecture that eliminates this bottleneck by directly inferring MI in a single forward pass. Pretrained on large-scale synthetic data with rich dependence patterns, InfoAtlas learns to identify diverse dependence structures and predict MI directly from the dataset. Comprehensive experiments demonstrate that InfoAtlas matches state-of-the-art neural estimators in accuracy while achieving $100\times$ speedup, can flexibly handle varying dimensions and sample sizes through a single unified model, and generalizes effectively to complex, real-world scenarios. By reformulating MI estimation as an inference task, InfoAtlas establishes a foundation for real-time dependency analysis.

Related papers

• Data-Efficient Deep Learning: Empirical Guidelines for Training Set Size Estimation in Inertial Sensor Classification [0.0]
  We introduce a unified framework that analyzes classification performance under both binary and multi-class scenarios.<n>We propose a quantitative stability point metric, defined as the sample size required for the learning curve to stabilize.<n>We offer a generalizable framework to extrapolate total data requirements from small-scale pilot studies.
  arXiv  Detail & Related papers  (2026-07-10T13:30:28Z)
• OpenSeisML: Open Large-Scale Real Seismic and well-log Dataset for Generative AI [1.379457310965574]
  We present OpenSeisML, a collection of real seismic to support generative AI (Gen-AI) for seismic inversion.<n>The datasets are curated from publicly available surveys in the UK National Data Repository (NDR)<n>We use checkshot data to establish the time-depth relationship and construct a velocity model through datasets for accurate conversion of post-stack seismic data.
  arXiv  Detail & Related papers  (2026-05-19T22:22:46Z)
• Earth System Foundation Model (ESFM): A unified framework for heterogeneous data integration and forecasting [52.53742727060265]
  Foundation models (FMs) for the Earth system learn statistical relationships between physical variables across massive datasets.<n>Here, we introduce Earth System Foundation Model (ESFM), a fully open model building on the 3Dwin UNet backbone of the pioneering model.<n>We present findings using dense gridded data (ERA5, CMIP6), regionally masked dense data, sparse gridded MODIS satellite data, and station data.<n>Results demonstrate competitive or superior performance relative to state-of-the-art benchmarks.
  arXiv  Detail & Related papers  (2026-04-20T11:40:39Z)
• Meta-Statistical Learning: Supervised Learning of Statistical Inference [59.463430294611626]
  This work demonstrates that the tools and principles driving the success of large language models (LLMs) can be repurposed to tackle distribution-level tasks.<n>We propose meta-statistical learning, a framework inspired by multi-instance learning that reformulates statistical inference tasks as supervised learning problems.
  arXiv  Detail & Related papers  (2025-02-17T18:04:39Z)
• A Benchmark Suite for Evaluating Neural Mutual Information Estimators on Unstructured Datasets [3.2228025627337864]
  Mutual Information (MI) is a fundamental metric for quantifying dependency between two random variables. This study introduces a comprehensive benchmark suite for evaluating neural MI estimators on unstructured datasets.
  arXiv  Detail & Related papers  (2024-10-14T14:22:38Z)
• PeFAD: A Parameter-Efficient Federated Framework for Time Series Anomaly Detection [51.20479454379662]
  We propose a. Federated Anomaly Detection framework named PeFAD with the increasing privacy concerns. We conduct extensive evaluations on four real datasets, where PeFAD outperforms existing state-of-the-art baselines by up to 28.74%.
  arXiv  Detail & Related papers  (2024-06-04T13:51:08Z)
• Exploring Federated Deep Learning for Standardising Naming Conventions in Radiotherapy Data [0.18749305679160366]
  Standardising structure volume names in radiotherapy (RT) data is necessary to enable data mining and analyses. No studies have considered that RT patient records are distributed across multiple data centres. This paper introduces a method that emulates real-world environments to establish standardised nomenclature. A multimodal deep artificial neural network was proposed to standardise RT data in federated settings.
  arXiv  Detail & Related papers  (2024-02-14T07:52:28Z)
• Federated Learning with Projected Trajectory Regularization [65.6266768678291]
  Federated learning enables joint training of machine learning models from distributed clients without sharing their local data. One key challenge in federated learning is to handle non-identically distributed data across the clients. We propose a novel federated learning framework with projected trajectory regularization (FedPTR) for tackling the data issue.
  arXiv  Detail & Related papers  (2023-12-22T02:12:08Z)
• Post-Estimation Smoothing: A Simple Baseline for Learning with Side Information [102.18616819054368]
  We propose a post-estimation smoothing operator as a fast and effective method for incorporating structural index data into prediction. Because the smoothing step is separate from the original predictor, it applies to a broad class of machine learning tasks. Our experiments on large scale spatial and temporal datasets highlight the speed and accuracy of post-estimation smoothing in practice.
  arXiv  Detail & Related papers  (2020-03-12T18:04:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.