Transformers with Physics-Informed Encodings and Simulation-Based Inference for Robust Detection of Eccentric Binary Black Holes in Pulsar Timing Array Data

• URL: http://arxiv.org/abs/2607.03904v1
• Date: Sat, 04 Jul 2026 14:59:03 GMT
• Title: Transformers with Physics-Informed Encodings and Simulation-Based Inference for Robust Detection of Eccentric Binary Black Holes in Pulsar Timing Array Data
• Abstract summary: We introduce a Transformer with a physics-informed positional-encoding framework for the efficient inference of eccentric binary black holes.<n>Our approach embeds analytical GW phase evolution directly into the model through structured positional encodings.<n>Across a range of signal-to-noise ratios, the proposed method achieves improved accuracy, sharper posteriors, and faster inference compared to physics-agnostic baselines.
• Score: 4.305160643026584
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Pulsar timing arrays (PTAs) provide a unique window into nanohertz gravitational waves (GWs), but extracting astrophysical parameters from noisy, long-baseline timing residuals remains computationally challenging with traditional Bayesian techniques due to the high dimensionality of the parameter space, complex and correlated noise models, and the cost of repeated likelihood evaluations. We introduce a Transformer with a physics-informed positional-encoding framework for the efficient inference of eccentric binary black holes in relativistic orbits from PTA data. Our approach embeds analytical GW phase evolution directly into the model through structured positional encodings, enabling the network to learn physically meaningful representations from raw PTA timing residuals. We then use generative models, including discrete and continuous conditional normalizing flows, to infer posterior distributions within a simulation-based inference framework. Across a range of signal-to-noise ratios, the proposed method achieves improved accuracy, sharper posteriors, and faster inference compared to physics-agnostic baselines. While presented for deterministic white-noise signals, the modular framework readily generalizes to realistic PTA analyses incorporating red noise and additional components. This work highlights the potential of physics-aware deep learning models as scalable alternatives to conventional inference pipelines for next-generation PTA datasets.

Related papers

• Learning Metamaterial Eigenmodes with Wavelet-Encoded Fourier Neural Operators [57.64362886310329]
  We show that a machine learning surrogate based on neural operators can learn and predict multiple eigenmodes of the elastic wave equation.<n>For metamaterial design, the resulting surrogate accelerates the simulation stage of the design cycle by three orders of magnitude relative to finite element analysis on a consumer-grade CPU.<n>These results also carry broader implications for designing input encodings in other multi-mode PDE solvers based on spectral neural operators.
  arXiv  Detail & Related papers  (2026-09-08T01:21:46Z)
• From Atoms to Entropy: Optimal Noise Allocation for Diffusion Training in the Convex Regime [52.459513910242485]
  We develop a general statistical framework for studying optimal noise-level allocation in diffusion training.<n>We show that the optimized training schedule admits an atomic minimizer, concentrated on finitely many noise levels.<n>We then evaluate the entropic schedule in larger-scale experiments, where full schedule optimization is currently intractable.
  arXiv  Detail & Related papers  (2026-07-10T20:59:06Z)
• Binary Black Hole Parameter Estimation with Hybrid CNN-Transformer Neural Networks [0.27998963147546146]
  Recent advances in Machine Learning and Deep Learning have sparked growing interest in their application to both signal detection and parameter estimation.<n>In this study, a hybrid Deep Learning strategy is proposed that leverages the effectiveness of Transformer encoders alongside well-established Convolutional Neural Network architectures.<n>The primary focus of this work is point estimation, producing single best-fit values for each parameter rather than full posterior distributions.
  arXiv  Detail & Related papers  (2026-06-11T22:05:33Z)
• Unveiling the Mechanism of Continuous Representation Full-Waveform Inversion: A Wave Based Neural Tangent Kernel Framework [52.632217600064344]
  Full-waveform inversion (FWI) estimates physical parameters in the wave equation from limited measurements.<n>FWI methods are limited by their notorious sensitivity to the accuracy of the initial models.<n>Recent progress in continuous representation FWI (CR-FWI) demonstrates that representing parameter models with a coordinate-based neural network, such as implicit neural representation (INR), can mitigate the dependence on initial models.
  arXiv  Detail & Related papers  (2026-03-23T03:22:33Z)
• Neutrino Oscillation Parameter Estimation Using Structured Hierarchical Transformers [2.9849405664643585]
  Neutrino oscillations encode fundamental information about neutrino masses and mixing parameters.<n>Traditional inference methods, such as likelihood-based or Monte Carlo sampling approaches, require extensive simulations to explore the parameter space.<n>We introduce a data-driven framework that reformulates atmospheric neutrino oscillation parameter inference as a supervised regression task.
  arXiv  Detail & Related papers  (2026-03-21T18:40:08Z)
• Efficient Real-Time Adaptation of ROMs for Unsteady Flows Using Data Assimilation [7.958594167693376]
  We propose an efficient retraining strategy for a parameterized Reduced Order Model (ROM)<n>The strategy attains accuracy comparable to full retraining while requiring only a fraction of the computational time.<n>We show that, for the dynamical system considered, the dominant source of error in out-of-sample forecasts stems from distortions of the latent manifold.
  arXiv  Detail & Related papers  (2026-02-26T16:43:28Z)
• From Sparse Sensors to Continuous Fields: STRIDE for Spatiotemporal Reconstruction [3.2580743227673694]
  We present STRIDE, a framework that maps high-dimensional spatial fields to a latent state with a temporaltemporal decoder.<n>We show that STRIDE supports super-resolution, supports super-resolution, and remains robust to noise.
  arXiv  Detail & Related papers  (2026-02-04T04:39:23Z)
• Learning from Scratch: Structurally-masked Transformer for Next Generation Lib-free Simulation [5.564681128355971]
  This paper proposes a neural framework for power and timing prediction of multi-stage data path.<n>To the best of our knowledge, this is the first language-based, netlist-aware neural network designed explicitly for standard cells.
  arXiv  Detail & Related papers  (2025-07-23T10:46:25Z)
• Efficient Transformed Gaussian Process State-Space Models for Non-Stationary High-Dimensional Dynamical Systems [49.819436680336786]
  We propose an efficient transformed Gaussian process state-space model (ETGPSSM) for scalable and flexible modeling of high-dimensional, non-stationary dynamical systems.<n>Specifically, our ETGPSSM integrates a single shared GP with input-dependent normalizing flows, yielding an expressive implicit process prior that captures complex, non-stationary transition dynamics.<n>Our ETGPSSM outperforms existing GPSSMs and neural network-based SSMs in terms of computational efficiency and accuracy.
  arXiv  Detail & Related papers  (2025-03-24T03:19:45Z)
• Online Variational Sequential Monte Carlo [49.97673761305336]
  We build upon the variational sequential Monte Carlo (VSMC) method, which provides computationally efficient and accurate model parameter estimation and Bayesian latent-state inference. Online VSMC is capable of performing efficiently, entirely on-the-fly, both parameter estimation and particle proposal adaptation.
  arXiv  Detail & Related papers  (2023-12-19T21:45:38Z)
• Real-time gravitational-wave science with neural posterior estimation [64.67121167063696]
  We demonstrate unprecedented accuracy for rapid gravitational-wave parameter estimation with deep learning. We analyze eight gravitational-wave events from the first LIGO-Virgo Gravitational-Wave Transient Catalog. We find very close quantitative agreement with standard inference codes, but with inference times reduced from O(day) to a minute per event.
  arXiv  Detail & Related papers  (2021-06-23T18:00:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.