Multi-Dataset Inverse Problem Solving with Distributed Generative AI

• URL: http://arxiv.org/abs/2608.26283v1
• Date: Wed, 26 Aug 2026 18:08:54 GMT
• Title: Multi-Dataset Inverse Problem Solving with Distributed Generative AI
• Abstract summary: We present a framework for simultaneously analyzing multiple heterogeneous datasets in the context of generative AI-based inverse problem solvers.<n>We validate the approach using a controlled setup inspired by a multi-detector scattering experiment.<n>The results show that our approach is well suited for real-world multi-dataset analyses in which experimental conditions vary across measurements.
• Score: 1.0877122445562903
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Extracting a shared set of unknown, not directly measurable quantities from multiple, heterogeneous datasets is a common challenge across scientific domains. A prominent example is the combination of datasets obtained from different measurements with different settings (e.g. varying detector resolutions). Analyzing such datasets jointly, rather than independently or after naive merging, is essential for obtaining precise and unbiased estimates of the unknowns, but requires careful treatment of dataset heterogeneity and is computationally demanding. We present a generalized framework for simultaneously analyzing multiple heterogeneous datasets in the context of generative AI-based inverse problem solvers. Building on our recent Scalable Asynchronous Generative Inverse Problem Solver (SAGIPS) framework, we extend the well-established distributed data-parallel training paradigm to non-identically distributed datasets, where each dataset is controlled by the same set of unknown inference parameters but covers a different region of the available feature space. Each dataset is processed through its own forward operator and discriminator, providing complementary constraints that collectively guide a shared generator toward global parameter consistency. We validate the approach using a controlled setup inspired by a multi-detector scattering experiment. We provide numerical evidence that our framework is robust to different data fidelities, which arise from unknown detector systematics in the Rutherford experiment, and we show the scaling behavior on multi-GPU leadership computing systems. The results show that our approach is well suited for real-world multi-dataset analyses in which experimental conditions vary across measurements.

Related papers

• Joint discovery of governing partial differential equations from multi-source datasets by competitive optimization [19.385544546206145]
  We present a competitive optimization framework designed to discover shared partial differential equations (PDEs) from multi-source datasets.<n>We demonstrate that fusing as few as 50 observations per dataset across seven cases recovers canonical equations with high accuracy.
  arXiv  Detail & Related papers  (2026-06-29T07:35:59Z)
• Towards One-for-All Anomaly Detection for Tabular Data [87.63505963517512]
  Tabular anomaly detection (TAD) is critical in many real-world applications.<n>We propose OFA-TAD, a framework that only requires one-time training on multiple source datasets.<n>Experiments on 34 datasets from 14 domains demonstrate that OFA-TAD achieves superior anomaly detection performance.
  arXiv  Detail & Related papers  (2026-03-15T14:42:05Z)
• Personalized Coupled Tensor Decomposition for Multimodal Data Fusion: Uniqueness and Algorithms [15.520496676392955]
  We introduce a personalized CTD framework tackling challenges of data fusion.<n>A flexible model is proposed where each dataset is represented as the sum of two components.<n>Two algorithms are proposed to compute the common and distinct components.
  arXiv  Detail & Related papers  (2024-12-02T04:19:47Z)
• PATH: A Discrete-sequence Dataset for Evaluating Online Unsupervised Anomaly Detection Approaches for Multivariate Time Series [0.01874930567916036]
  Benchmarking anomaly detection approaches for multivariate time series is a challenging task due to a lack of high-quality datasets.<n>We propose a solution: a diverse, extensive, and non-trivial dataset generated via state-of-the-art simulation tools.<n>Our dataset represents a discrete-sequence problem, which remains unaddressed by previously-proposed solutions in literature.
  arXiv  Detail & Related papers  (2024-11-21T09:03:12Z)
• Cross-Dataset Gaze Estimation by Evidential Inter-intra Fusion [38.93368421481945]
  We propose a novel Evidential Inter-intra Fusion EIF framework for training a cross-dataset model.<n>We build independent single-dataset branches for various datasets.<n>We further create a cross-dataset branch to integrate the generalizable features from single-dataset branches.
  arXiv  Detail & Related papers  (2024-09-07T08:53:17Z)
• AgentOhana: Design Unified Data and Training Pipeline for Effective Agent Learning [98.26836657967162]
  textbfAgentOhana aggregates agent trajectories from distinct environments, spanning a wide array of scenarios. textbfxLAM-v0.1, a large action model tailored for AI agents, demonstrates exceptional performance across various benchmarks.
  arXiv  Detail & Related papers  (2024-02-23T18:56:26Z)
• Robust Detection of Lead-Lag Relationships in Lagged Multi-Factor Models [61.10851158749843]
  Key insights can be obtained by discovering lead-lag relationships inherent in the data. We develop a clustering-driven methodology for robust detection of lead-lag relationships in lagged multi-factor models.
  arXiv  Detail & Related papers  (2023-05-11T10:30:35Z)
• Learning to Bound Counterfactual Inference in Structural Causal Models from Observational and Randomised Data [64.96984404868411]
  We derive a likelihood characterisation for the overall data that leads us to extend a previous EM-based algorithm. The new algorithm learns to approximate the (unidentifiability) region of model parameters from such mixed data sources. It delivers interval approximations to counterfactual results, which collapse to points in the identifiable case.
  arXiv  Detail & Related papers  (2022-12-06T12:42:11Z)
• Deep Federated Anomaly Detection for Multivariate Time Series Data [93.08977495974978]
  We present a Federated Exemplar-based Deep Neural Network (Fed-ExDNN) to conduct anomaly detection for multivariate time series data on different edge devices. We show that ExDNN and Fed-ExDNN can outperform state-of-the-art anomaly detection algorithms and federated learning techniques.
  arXiv  Detail & Related papers  (2022-05-09T05:06:58Z)
• Equivariance Allows Handling Multiple Nuisance Variables When Analyzing Pooled Neuroimaging Datasets [53.34152466646884]
  In this paper, we show how bringing recent results on equivariant representation learning instantiated on structured spaces together with simple use of classical results on causal inference provides an effective practical solution. We demonstrate how our model allows dealing with more than one nuisance variable under some assumptions and can enable analysis of pooled scientific datasets in scenarios that would otherwise entail removing a large portion of the samples.
  arXiv  Detail & Related papers  (2022-03-29T04:54:06Z)
• Multimodal Data Fusion in High-Dimensional Heterogeneous Datasets via Generative Models [16.436293069942312]
  We are interested in learning probabilistic generative models from high-dimensional heterogeneous data in an unsupervised fashion. We propose a general framework that combines disparate data types through the exponential family of distributions. The proposed algorithm is presented in detail for the commonly encountered heterogeneous datasets with real-valued (Gaussian) and categorical (multinomial) features.
  arXiv  Detail & Related papers  (2021-08-27T18:10:31Z)
• Learning while Respecting Privacy and Robustness to Distributional Uncertainties and Adversarial Data [66.78671826743884]
  The distributionally robust optimization framework is considered for training a parametric model. The objective is to endow the trained model with robustness against adversarially manipulated input data. Proposed algorithms offer robustness with little overhead.
  arXiv  Detail & Related papers  (2020-07-07T18:25:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.