TILT: Target-induced loss tilting under covariate shift

• URL: http://arxiv.org/abs/2605.14280v1
• Date: Thu, 14 May 2026 02:26:34 GMT
• Title: TILT: Target-induced loss tilting under covariate shift
• Abstract summary: We introduce and analyze Target-Induced Loss Tilting (TILT) for unsupervised domain adaptation.<n>Based on a novel objective function that decomposes the source predictor as $f+b$, fits $f+b$ on labeled source data while simultaneously penalizing the auxiliary component $b$ on unlabeled target inputs.<n>We show that this target-side penalty implicitly induces relative importance weighting at the population level, but in terms of an estimand $b*_f$ that is self-localized to the current error, and remains uniformly bounded for any source-target pair.
• Score: 14.526961596109361
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce and analyze Target-Induced Loss Tilting (TILT) for unsupervised domain adaptation under covariate shift. It is based on a novel objective function that decomposes the source predictor as $f+b$, fits $f+b$ on labeled source data while simultaneously penalizing the auxiliary component $b$ on unlabeled target inputs. The resulting fit $f$ is deployed as the final target predictor. At the population level, we show that this target-side penalty implicitly induces relative importance weighting at the population level, but in terms of an estimand $b^*_f$ that is self-localized to the current error, and remains uniformly bounded for any source-target pair (even those with disjoint supports). We prove a general finite-sample oracle inequality on the excess risk, and use it to give an end-to-end guarantee for training with sparse ReLU networks. Experiments on controlled regression problems and shifted CIFAR-100 distillation show that TILT improves target-domain performance over source-only training, exact importance weighting, and relative density-ratio baselines, with a stable dependence on the regularization parameter.

Related papers

• Beyond Homoscedasticity: Decoupled Uncertainty Optimization for Deep Imbalanced Regression [7.011984503405774]
  Deep Imbalanced Regression is pervasive in continuous prediction tasks across diverse modalities.<n>We propose DUO, an uncertainty-aware long-tailed regression framework.<n>We show that DUO achieves the best few-shot bMAE and GM on IMDB-WIKI-DIR, AgeDB-DIR, and AAV2-DIR while remaining competitive on few-shot MAE.
  arXiv  Detail & Related papers  (2026-09-04T11:06:57Z)
• Convex-Hull-Neighborhood Smooth Dual Generalization: Controlling Local Correction Propagation in Offline RL [7.803284875836737]
  bootstrapping can amplify estimation errors in offline reinforcement learning.<n>We propose Convex Hull Neighborhood Smooth Dual Generalization, which expresses the Bellman backup as an in-sample value target plus a CHN-local correction.<n> Experiments on Gym-MuJoCo and AntMaze show strong aggregate performance and stable value estimation.
  arXiv  Detail & Related papers  (2026-08-04T04:30:02Z)
• How Useful is Causal Invariance for Domain Adaptation in Finite-Sample Settings? [58.740078141879984]
  Machine learning models often degrade when they are deployed on a target distribution that differs from the source distributions they were trained on.<n>Recent work in causality-based domain generalization has shown how shared causal structure can induce invariant predictors.<n>In this paper, we study when full or partial causal knowledge can provably improve supervised domain adaptation.
  arXiv  Detail & Related papers  (2026-06-10T21:07:49Z)
• When to Trust Confidence Thresholding: Calibration Diagnostics for Pseudo-Labelled Regression [1.14219428942199]
  A standard practice is to threshold the calibrated score at a confidence cutoff and treat the hard label as the truth.<n>We develop a calibration-aware diagnostic apparatus for pseudo-labelling pipelines.<n>We derive a closed-form expression for the attenuation bias that confidence thresholding induces in the downstream regression coefficient.
  arXiv  Detail & Related papers  (2026-05-12T21:49:11Z)
• Adaptive Threshold-Driven Continuous Greedy Method for Scalable Submodular Optimization [0.0]
  Submodular under matroid constraints is a fundamental problem in optimization with applications in sensing, data-marization, active learning, and resource allocation.<n>We propose textitATCG (underlineAdaptive underlineThresholded underlineThresholded underlineGreedy), expanding each agent's active set only when current candidates fail to capture sufficient marginal gain.<n>Experiments on a classbalanced prototype selection problem over a CIFAR-10 animal dataset show that textit
  arXiv  Detail & Related papers  (2026-04-03T19:32:39Z)
• Improving Search Agent with One Line of Code [68.58667107354253]
  Tool-based Agentic Reinforcement Learning (TARL) has emerged as a promising paradigm for training search agents to interact with external tools.<n>We propose textbfSearch textbfAgent textbfPolicy textbfOptimization (textbfSAPO), which stabilizes training via a conditional token-level KL constraint.
  arXiv  Detail & Related papers  (2026-03-10T04:07:39Z)
• Coverage Guarantees for Pseudo-Calibrated Conformal Prediction under Distribution Shift [1.5861469511290378]
  Conformal prediction offers marginal coverage guarantees if the data distribution shifts.<n>We analyze the use of pseudo-calibration as a tool to counter this performance loss.<n>We propose a source-tuned pseudo-calibration algorithm that interpolates between hard pseudo-labels and randomized labels.
  arXiv  Detail & Related papers  (2026-02-16T16:48:39Z)
• Non-Stationary Online Structured Prediction with Surrogate Losses [22.848052937383244]
  We prove a bound of the form $F_T + C (1 + P_T)$ on the cumulative target loss.<n>Our core idea is to synthesize the dynamic regret bound of the gradient online descent (OGD) with the technique of exploiting the surrogate gap.<n>Our analysis also sheds light on a new Polyak-style learning rate for OGD, which systematically offers target-loss guarantees and exhibits promising empirical performance.
  arXiv  Detail & Related papers  (2025-10-08T14:43:44Z)
• The Role of Baselines in Policy Gradient Optimization [83.42050606055822]
  We show that the emphstate value baseline allows on-policy. emphnatural policy gradient (NPG) to converge to a globally optimal. policy at an $O (1/t) rate gradient. We find that the primary effect of the value baseline is to textbfreduce the aggressiveness of the updates rather than their variance.
  arXiv  Detail & Related papers  (2023-01-16T06:28:00Z)
• Divide and Contrast: Source-free Domain Adaptation via Adaptive Contrastive Learning [122.62311703151215]
  Divide and Contrast (DaC) aims to connect the good ends of both worlds while bypassing their limitations. DaC divides the target data into source-like and target-specific samples, where either group of samples is treated with tailored goals. We further align the source-like domain with the target-specific samples using a memory bank-based Maximum Mean Discrepancy (MMD) loss to reduce the distribution mismatch.
  arXiv  Detail & Related papers  (2022-11-12T09:21:49Z)
• The Power and Limitation of Pretraining-Finetuning for Linear Regression under Covariate Shift [127.21287240963859]
  We investigate a transfer learning approach with pretraining on the source data and finetuning based on the target data. For a large class of linear regression instances, transfer learning with $O(N2)$ source data is as effective as supervised learning with $N$ target data.
  arXiv  Detail & Related papers  (2022-08-03T05:59:49Z)
• Test-time Batch Statistics Calibration for Covariate Shift [66.7044675981449]
  We propose to adapt the deep models to the novel environment during inference. We present a general formulation $alpha$-BN to calibrate the batch statistics. We also present a novel loss function to form a unified test time adaptation framework Core.
  arXiv  Detail & Related papers  (2021-10-06T08:45:03Z)
• KL Guided Domain Adaptation [88.19298405363452]
  Domain adaptation is an important problem and often needed for real-world applications. A common approach in the domain adaptation literature is to learn a representation of the input that has the same distributions over the source and the target domain. We show that with a probabilistic representation network, the KL term can be estimated efficiently via minibatch samples.
  arXiv  Detail & Related papers  (2021-06-14T22:24:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.