A Three-Stage Bayesian Transfer Learning Framework to Improve Predictions in Data-Scarce Domains

• URL: http://arxiv.org/abs/2510.26541v1
• Date: Thu, 30 Oct 2025 14:30:53 GMT
• Title: A Three-Stage Bayesian Transfer Learning Framework to Improve Predictions in Data-Scarce Domains
• Authors: Aidan Furlong, Robert Salko, Xingang Zhao, Xu Wu,
• Abstract summary: Domain-adversarial neural networks (DANNs) help improve transfer under greater domain shifts in a semi-supervised setting.<n>This study introduces a fully-supervised three-stage framework, the staged Bayesian domain-adversarial neural network (staged B-DANN)<n>The results of this study show that the staged B-DANN method can improve predictive accuracy and generalization, potentially assisting other domains in nuclear engineering.
• Score: 6.7949074631455995
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The use of ML in engineering has grown steadily to support a wide array of applications. Among these methods, deep neural networks have been widely adopted due to their performance and accessibility, but they require large, high-quality datasets. Experimental data are often sparse, noisy, or insufficient to build resilient data-driven models. Transfer learning, which leverages relevant data-abundant source domains to assist learning in data-scarce target domains, has shown efficacy. Parameter transfer, where pretrained weights are reused, is common but degrades under large domain shifts. Domain-adversarial neural networks (DANNs) help address this issue by learning domain-invariant representations, thereby improving transfer under greater domain shifts in a semi-supervised setting. However, DANNs can be unstable during training and lack a native means for uncertainty quantification. This study introduces a fully-supervised three-stage framework, the staged Bayesian domain-adversarial neural network (staged B-DANN), that combines parameter transfer and shared latent space adaptation. In Stage 1, a deterministic feature extractor is trained on the source domain. This feature extractor is then adversarially refined using a DANN in Stage 2. In Stage 3, a Bayesian neural network is built on the adapted feature extractor for fine-tuning on the target domain to handle conditional shifts and yield calibrated uncertainty estimates. This staged B-DANN approach was first validated on a synthetic benchmark, where it was shown to significantly outperform standard transfer techniques. It was then applied to the task of predicting critical heat flux in rectangular channels, leveraging data from tube experiments as the source domain. The results of this study show that the staged B-DANN method can improve predictive accuracy and generalization, potentially assisting other domains in nuclear engineering.

Related papers

• Topology-Aware Modeling for Unsupervised Simulation-to-Reality Point Cloud Recognition [63.55828203989405]
  We introduce a novel Topology-Aware Modeling (TAM) framework for Sim2Real UDA on object point clouds.<n>Our approach mitigates the domain gap by leveraging global spatial topology, characterized by low-level, high-frequency 3D structures.<n>We propose an advanced self-training strategy that combines cross-domain contrastive learning with self-training.
  arXiv  Detail & Related papers  (2025-06-26T11:53:59Z)
• Dual-Path Adversarial Lifting for Domain Shift Correction in Online Test-time Adaptation [59.18151483767509]
  We introduce a dual-path token lifting for domain shift correction in test time adaptation. We then perform dual-path lifting with interleaved token prediction and update between the path of domain shift tokens and the path of class tokens. Experimental results on the benchmark datasets demonstrate that our proposed method significantly improves the online fully test-time domain adaptation performance.
  arXiv  Detail & Related papers  (2024-08-26T02:33:47Z)
• Adapting to Distribution Shift by Visual Domain Prompt Generation [34.19066857066073]
  We adapt a model at test-time using a few unlabeled data to address distribution shifts. We build a knowledge bank to learn the transferable knowledge from source domains. The proposed method outperforms previous work on 5 large-scale benchmarks including WILDS and DomainNet.
  arXiv  Detail & Related papers  (2024-05-05T02:44:04Z)
• Test-Time Domain Adaptation by Learning Domain-Aware Batch Normalization [39.14048972373775]
  Test-time domain adaptation aims to adapt the model trained on source domains to unseen target domains using a few unlabeled images. Previous works normally update the whole network naively without explicitly decoupling the knowledge between label and domain. We propose to reduce such learning interference and elevate the domain knowledge learning by only manipulating the BN layer.
  arXiv  Detail & Related papers  (2023-12-15T19:22:21Z)
• Robust Representation Learning with Self-Distillation for Domain Generalization [2.0817769887373245]
  We propose a novel domain generalization technique called Robust Representation Learning with Self-Distillation. We observe an average accuracy improvement in the range of 1.2% to 2.3% over the state-of-the-art on three datasets.
  arXiv  Detail & Related papers  (2023-02-14T07:39:37Z)
• Source-Free Domain Adaptation via Distribution Estimation [106.48277721860036]
  Domain Adaptation aims to transfer the knowledge learned from a labeled source domain to an unlabeled target domain whose data distributions are different. Recently, Source-Free Domain Adaptation (SFDA) has drawn much attention, which tries to tackle domain adaptation problem without using source data. In this work, we propose a novel framework called SFDA-DE to address SFDA task via source Distribution Estimation.
  arXiv  Detail & Related papers  (2022-04-24T12:22:19Z)
• Self-supervised Autoregressive Domain Adaptation for Time Series Data [9.75443057146649]
  Unsupervised domain adaptation (UDA) has successfully addressed the domain shift problem for visual applications. These approaches may have limited performance for time series data due to the following reasons. We propose a Self-supervised Autoregressive Domain Adaptation (SLARDA) framework to address these limitations.
  arXiv  Detail & Related papers  (2021-11-29T08:17:23Z)
• Stagewise Unsupervised Domain Adaptation with Adversarial Self-Training for Road Segmentation of Remote Sensing Images [93.50240389540252]
  Road segmentation from remote sensing images is a challenging task with wide ranges of application potentials. We propose a novel stagewise domain adaptation model called RoadDA to address the domain shift (DS) issue in this field. Experiment results on two benchmarks demonstrate that RoadDA can efficiently reduce the domain gap and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-08-28T09:29:14Z)
• Coarse to Fine: Domain Adaptive Crowd Counting via Adversarial Scoring Network [58.05473757538834]
  This paper proposes a novel adversarial scoring network (ASNet) to bridge the gap across domains from coarse to fine granularity. Three sets of migration experiments show that the proposed methods achieve state-of-the-art counting performance.
  arXiv  Detail & Related papers  (2021-07-27T14:47:24Z)
• Unsupervised Domain Adaptive 3D Detection with Multi-Level Consistency [90.71745178767203]
  Deep learning-based 3D object detection has achieved unprecedented success with the advent of large-scale autonomous driving datasets. Existing 3D domain adaptive detection methods often assume prior access to the target domain annotations, which is rarely feasible in the real world. We study a more realistic setting, unsupervised 3D domain adaptive detection, which only utilizes source domain annotations.
  arXiv  Detail & Related papers  (2021-07-23T17:19:23Z)
• A Curriculum-style Self-training Approach for Source-Free Semantic Segmentation [91.13472029666312]
  We propose a curriculum-style self-training approach for source-free domain adaptive semantic segmentation. Our method yields state-of-the-art performance on source-free semantic segmentation tasks for both synthetic-to-real and adverse conditions.
  arXiv  Detail & Related papers  (2021-06-22T10:21:39Z)
• Neural Supervised Domain Adaptation by Augmenting Pre-trained Models with Random Units [14.183224769428843]
  Neural Transfer Learning (TL) is becoming ubiquitous in Natural Language Processing (NLP) In this paper, we show through interpretation methods that such scheme, despite its efficiency, is suffering from a main limitation. We propose to augment the pre-trained model with normalised, weighted and randomly initialised units that foster a better adaptation while maintaining the valuable source knowledge.
  arXiv  Detail & Related papers  (2021-06-09T09:29:11Z)
• Neuron Linear Transformation: Modeling the Domain Shift for Crowd Counting [34.560447389853614]
  Cross-domain crowd counting (CDCC) is a hot topic due to its importance in public safety. We propose a Neuron Linear Transformation (NLT) method, exploiting domain factor and bias weights to learn the domain shift. Extensive experiments and analysis on six real-world datasets validate that NLT achieves top performance.
  arXiv  Detail & Related papers  (2020-04-05T09:15:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.