Statistical Embeddings for Similarity, Retrieval, and Interpretable Alignment of Numeric Tabular Datasets

• URL: http://arxiv.org/abs/2605.30289v1
• Date: Thu, 28 May 2026 17:40:42 GMT
• Title: Statistical Embeddings for Similarity, Retrieval, and Interpretable Alignment of Numeric Tabular Datasets
• Abstract summary: The proposed methodology characterizes numeric datasets through structured exploratory data analysis descriptors.<n>A penalized formulation of Canonical Correlation Analysis (CCA) is applied to recover sparse, interpretable variable-level correspondences.<n>The methodology is evaluated across 15 datasets spanning general-purpose benchmarks, materials informatics, and nuclear-grade graphite characterization.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Numeric tabular datasets are the dominant data format in scientific practice, yet large language models lack native mechanisms for representing numeric datasets in a meaningful way across heterogeneous feature spaces. Existing approaches either target predictive modeling over individual datasets, which requires a shared set of variable definitions, or lack mechanisms for interpretable cross-dataset alignment. The proposed methodology characterizes numeric tabular datasets through structured exploratory data analysis descriptors, embeds those descriptors into a shared vector space using a pretrained sentence transformer, and quantifies cross-dataset similarity via Canonical Correlation Analysis (CCA). Furthermore, a penalized formulation of CCA is applied to recover sparse, interpretable variable-level correspondences between datasets, identifying which statistical descriptors or variable-level quantities drive cross-dataset alignment without requiring shared variable names or feature conventions. Differential privacy is optionally applied to the descriptor set prior to embedding, supporting deployment in sensitive data contexts without requiring access to raw observations at time of comparison. The methodology is evaluated across 15 datasets spanning general-purpose benchmarks, materials informatics, and nuclear-grade graphite characterization. Results demonstrate a total P@1 score of 0.9, with known nearest-neighbor retrieval and cluster structure remaining robust across embedding ablations and differential privacy budgets. The proposed framework provides a principled pathway for integrating heterogeneous numeric data into retrieval-augmented generation pipelines while preserving statistical context, with direct applications to data-driven algorithm selection and simulation model initialization for unknown datasets.

Related papers

• Copula Adapted Directed Acyclic Graph for Cluster Representation of Biomedical Data [0.0]
  Stratifying unlabeled biomedical data based on complex relationships between features eliminates the need for data labels.<n>Traditional clustering methods assume restrictive data distributions, making them suboptimal for capturing complex dependencies.<n>This paper introduces a novel cluster-friendly data presentation framework that integrates the non-Gaussian and non-linear feature dependence of copula models.
  arXiv  Detail & Related papers  (2026-09-14T19:07:53Z)
• Data Evolution by Wittgenstein's Rule Following [0.08594140167290097]
  Wittgenstein's Rule Following (WRF) is a framework for evolving or generating a new dataset from a sequence of previously observed datasets.<n>WRF represents each dataset by structural descriptors rather than pointwise correspondences.<n>The proposed framework allows both sample size and feature dimension to vary over time and does not assume that the next dataset is a direct transformation of the last one.
  arXiv  Detail & Related papers  (2026-06-21T21:18:55Z)
• Active Learning with Low-Rank Structure for Data Selection [85.43185363043362]
  We introduce a new data selection framework based on low-rank approximation and residual-based sampling.<n>We show that it is possible to select a weighted subset of $tildeOleft(k + frac1varepsilon2right)$ data points whose average loss approximates the average loss over the full dataset.
  arXiv  Detail & Related papers  (2026-06-14T22:29:59Z)
• idSCD: Identifying Training Datasets through Semantic Correlation Descriptors [5.756453494739546]
  We argue that datasets leave dataset-specific traces in a model's learned semantic correlation structure.<n>We introduce a white-box semantic fingerprinting approach based on semantic correlation descriptors (SCDs)<n>We propose a practical SCD-based membership score that tests whether a target dataset is part of a model's training mixture.
  arXiv  Detail & Related papers  (2026-05-28T18:38:15Z)
• Beyond Statistical Co-occurrence: Unlocking Intrinsic Semantics for Tabular Data Clustering [52.97486694817375]
  This paper proposes Tabular-Augmented Contrastive Clustering (TagCC) to bridge the gap between dataset-specific statistics and intrinsic semantic knowledge.<n>TagCC distills underlying data semantics into textual anchors via semantic-aware transformation.<n>It is jointly optimized with a clustering objective, ensuring that the learned representations are both semantically coherent and clustering-friendly.
  arXiv  Detail & Related papers  (2026-04-13T00:25:22Z)
• Bridging the Semantic Gap for Categorical Data Clustering via Large Language Models [64.58262227709842]
  ARISE (Attention-weighted Representation with Integrated Semantic Embeddings) is presented.<n>It builds semantic-aware representations that complement the metric space of categorical data for accurate clustering.<n>Experiments on eight benchmark datasets demonstrate consistent improvements over seven representative counterparts.
  arXiv  Detail & Related papers  (2026-01-03T11:37:46Z)
• A Dataset for Semantic Segmentation in the Presence of Unknowns [49.795683850385956]
  Existing datasets allow evaluation of only knowns or unknowns - but not both.<n>We propose a novel anomaly segmentation dataset, ISSU, that features a diverse set of anomaly inputs from cluttered real-world environments.<n>The dataset is twice larger than existing anomaly segmentation datasets.
  arXiv  Detail & Related papers  (2025-03-28T10:31:01Z)
• Downstream-Pretext Domain Knowledge Traceback for Active Learning [138.02530777915362]
  We propose a downstream-pretext domain knowledge traceback (DOKT) method that traces the data interactions of downstream knowledge and pre-training guidance. DOKT consists of a traceback diversity indicator and a domain-based uncertainty estimator. Experiments conducted on ten datasets show that our model outperforms other state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-07-20T01:34:13Z)
• Tunable correlation retention: A statistical method for generating synthetic data [0.0]
  We propose a method to generate statistically representative synthetic data from a given dataset.<n>The main goal of our method is for the created data to mimic the inter--feature correlations present in the original data.<n>We evaluate the method by comparing the datasets using the Pearson correlation matrix with different levels of resolution and depths of correlation.
  arXiv  Detail & Related papers  (2024-03-03T10:35:46Z)
• Comparative Study of Inference Methods for Interpolative Decomposition [4.913248451323163]
  We propose a probabilistic model with automatic relevance determination (ARD) for learning interpolative decomposition (ID) We evaluate the model on a variety of real-world datasets including CCLE $EC50$, CCLE $IC50$, Gene Body Methylation, and Promoter Methylation datasets with different sizes, and dimensions.
  arXiv  Detail & Related papers  (2022-06-29T11:37:05Z)
• Dynamic Kernel Matching for Non-conforming Data: A Case Study of T-cell Receptor Datasets [0.0]
  We describe an approach for modifying established statistical classifiers to handle non-conforming data. We consider (i) a dataset of T-cell receptor (TCR) sequences labelled by disease antigen and (ii) a dataset of sequenced TCR repertoires labelled by patient cytomegalovirus (CMV) serostatus.
  arXiv  Detail & Related papers  (2021-03-18T18:39:14Z)
• Asymptotic Analysis of an Ensemble of Randomly Projected Linear Discriminants [94.46276668068327]
  In [1], an ensemble of randomly projected linear discriminants is used to classify datasets. We develop a consistent estimator of the misclassification probability as an alternative to the computationally-costly cross-validation estimator. We also demonstrate the use of our estimator for tuning the projection dimension on both real and synthetic data.
  arXiv  Detail & Related papers  (2020-04-17T12:47:04Z)
• New advances in enumerative biclustering algorithms with online partitioning [80.22629846165306]
  This paper further extends RIn-Close_CVC, a biclustering algorithm capable of performing an efficient, complete, correct and non-redundant enumeration of maximal biclusters with constant values on columns in numerical datasets. The improved algorithm is called RIn-Close_CVC3, keeps those attractive properties of RIn-Close_CVC, and is characterized by: a drastic reduction in memory usage; a consistent gain in runtime.
  arXiv  Detail & Related papers  (2020-03-07T14:54:26Z)
• Adaptive Discrete Smoothing for High-Dimensional and Nonlinear Panel Data [4.550919471480445]
  We develop a data-driven smoothing technique for high-dimensional and non-linear panel data models. The weights are determined by a data-driven way and depend on the similarity between the corresponding functions. We conduct a simulation study which shows that the prediction can be greatly improved by using our estimator.
  arXiv  Detail & Related papers  (2019-12-30T09:50:58Z)
• Meta-analysis of heterogeneous data: integrative sparse regression in high-dimensions [21.162280861396205]
  We consider the task of meta-analysis in high-dimensional settings in which the data sources are similar but non-identical. We introduce a global parameter that emphasizes interpretability and statistical efficiency in the presence of heterogeneity. We demonstrate the benefits of our approach on a large-scale drug treatment dataset involving several different cancer cell-lines.
  arXiv  Detail & Related papers  (2019-12-26T20:30:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.