RADAR: Relative Angular Divergence Across Representations

• URL: http://arxiv.org/abs/2605.23028v1
• Date: Thu, 21 May 2026 20:51:00 GMT
• Title: RADAR: Relative Angular Divergence Across Representations
• Abstract summary: We propose RADAR, a simple, geometrically grounded metric for estimating cross-domain transferability in foundation models.<n>We evaluate the metric across multiple modalities, including cross-lingual sentiment classification with text embedding models and cross-domain image classification with foundation vision models.<n>Our ablations suggest that the effectiveness of transferability estimation depends on the geometry of the model's internal representation space.
• Score: 4.603755953026689
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine learning methods rely on data. However, gathering suitable data can be challenging due to availability constraints, cost, or the need for domain expertise. Expanding datasets with additional sources is a common response to limited data, yet this practice does not always improve downstream performance and can sometimes lead to a loss of performance, known as negative transfer. We propose RADAR, a simple, geometrically grounded metric for estimating cross-domain transferability in foundation models. RADAR analyzes the layer-wise evolution of representations by measuring angular alignments and relative changes in distance along layer-to-layer displacement trajectories, and by comparing empirical distributions of within-domain and cross-domain dynamics. We hypothesize that domain transferability is related to the divergence between these trajectory distributions. We evaluate the metric across multiple modalities, including cross-lingual sentiment classification with text embedding models and cross-domain image classification with foundation vision models. Across several settings, RADAR provides competitive predictive performance relative to existing transferability metrics on several vision and text benchmarks, with particularly strong results when domain transitions are smooth or cleanly separated. Our ablations further suggest that the effectiveness of transferability estimation depends on the geometry of the model's internal representation space, with different modalities favoring different topological formulations.

Related papers

• Multimodal Foundation Models Adaptation based on Domain-Aware Relaxed Orthogonal Subspace for Remote Sensing [16.45880716714667]
  Domain-aware Relaxed Orthogonal Subspace adaptation (DROS)<n>DROS reformulates low-rank adaptation as data-conditioned subspace learning and flexible subspace adaptation.<n>Experiments on multiple remote sensing benchmarks demonstrate that DROS achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-09-12T02:19:11Z)
• OmniLiDAR: A Unified Diffusion Framework for Multi-Domain 3D LiDAR Generation [100.36917520403227]
  We present a unified text-conditioned diffusion framework that generates LiDAR scans in a shared range-image representation across eight representative domains.<n>In the absence of a public consolidated benchmark, we construct an 8-domain dataset by combining real-world scans with physically based weather simulation and systematic beam reduction.
  arXiv  Detail & Related papers  (2026-05-13T17:42:20Z)
• COP-GEN: Latent Diffusion Transformer for Copernicus Earth Observation Data -- Generation Stochastic by Design [9.278432103577925]
  Earth observation applications increasingly rely on data from multiple sensors, including optical, radar, elevation, and land-cover products.<n>We introduce COP-GEN, a latent diffusion transformer that models the joint distribution of heterogeneous Earth Observation modalities at their native spatial resolutions.<n>Experiments on a large-scale global multimodal dataset show that COP-GEN generates diverse yet physically consistent realisations while maintaining strong peak fidelity across optical, radar, and elevation modalities.
  arXiv  Detail & Related papers  (2026-03-03T18:31:46Z)
• Minimax optimal adaptive structured transfer learning through semi-parametric domain-varying coefficient model [9.091986429838117]
  We study a multi-source, single-target transfer learning problem under conditional distributional drift.<n>We develop an adaptive transfer learning estimator that selectively borrows strength from informative source domains.
  arXiv  Detail & Related papers  (2026-02-20T03:53:06Z)
• Nonparametric Data Attribution for Diffusion Models [57.820618036556084]
  Data attribution for generative models seeks to quantify the influence of individual training examples on model outputs.<n>We propose a nonparametric attribution method that operates entirely on data, measuring influence via patch-level similarity between generated and training images.
  arXiv  Detail & Related papers  (2025-10-16T03:37:16Z)
• DIDS: Domain Impact-aware Data Sampling for Large Language Model Training [61.10643823069603]
  We present Domain Impact-aware Data Sampling (DIDS) for large language models.<n>DIDS group training data based on learning effects, where a proxy language model and dimensionality reduction are employed.<n>It achieves 3.4% higher average performance while maintaining comparable training efficiency.
  arXiv  Detail & Related papers  (2025-04-17T13:09:38Z)
• xTED: Cross-Domain Adaptation via Diffusion-Based Trajectory Editing [21.37585797507323]
  Cross-domain policy transfer methods mostly aim at learning domain correspondences or corrections to facilitate policy learning.<n>We propose the Cross-Domain Trajectory EDiting framework that employs a specially designed diffusion model for cross-domain trajectory adaptation.<n>Our proposed model architecture effectively captures the intricate dependencies among states, actions, and rewards, as well as the dynamics patterns within target data.
  arXiv  Detail & Related papers  (2024-09-13T10:07:28Z)
• Improving Transferability for Cross-domain Trajectory Prediction via Neural Stochastic Differential Equation [41.09061877498741]
  discrepancies exist among datasets due to external factors and data acquisition strategies. The proficient performance of models trained on large-scale datasets has limited transferability on other small-size datasets. We propose a method based on continuous and utilization of Neural Differential Equations (NSDE) for alleviating discrepancies. The effectiveness of our method is validated against state-of-the-art trajectory prediction models on the popular benchmark datasets: nuScenes, Argoverse, Lyft, INTERACTION, and Open Motion dataset.
  arXiv  Detail & Related papers  (2023-12-26T06:50:29Z)
• SALUDA: Surface-based Automotive Lidar Unsupervised Domain Adaptation [62.889835139583965]
  We introduce an unsupervised auxiliary task of learning an implicit underlying surface representation simultaneously on source and target data. As both domains share the same latent representation, the model is forced to accommodate discrepancies between the two sources of data. Our experiments demonstrate that our method achieves a better performance than the current state of the art, both in real-to-real and synthetic-to-real scenarios.
  arXiv  Detail & Related papers  (2023-04-06T17:36:23Z)
• VTAE: Variational Transformer Autoencoder with Manifolds Learning [144.0546653941249]
  Deep generative models have demonstrated successful applications in learning non-linear data distributions through a number of latent variables. The nonlinearity of the generator implies that the latent space shows an unsatisfactory projection of the data space, which results in poor representation learning. We show that geodesics and accurate computation can substantially improve the performance of deep generative models.
  arXiv  Detail & Related papers  (2023-04-03T13:13:19Z)
• Relation Matters: Foreground-aware Graph-based Relational Reasoning for Domain Adaptive Object Detection [81.07378219410182]
  We propose a new and general framework for DomainD, named Foreground-aware Graph-based Reasoning (FGRR) FGRR incorporates graph structures into the detection pipeline to explicitly model the intra- and inter-domain foreground object relations. Empirical results demonstrate that the proposed FGRR exceeds the state-of-the-art on four DomainD benchmarks.
  arXiv  Detail & Related papers  (2022-06-06T05:12:48Z)
• Geometry-Aware Unsupervised Domain Adaptation [12.298214579392129]
  Unsupervised Domain Adaptation (UDA) aims to transfer the knowledge from the labeled source domain to the unlabeled target domain in the presence of dataset shift. Most existing methods cannot address the domain alignment and class discrimination well, which may distort the intrinsic data structure for downstream tasks. We propose a novel geometry-aware model to learn the transferability and discriminability simultaneously via nuclear norm optimization.
  arXiv  Detail & Related papers  (2021-12-21T08:45:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.