Refnd: Preventing Data Leakage in Relational Datasets

• URL: http://arxiv.org/abs/2607.19376v1
• Date: Tue, 30 Jun 2026 19:53:33 GMT
• Title: Refnd: Preventing Data Leakage in Relational Datasets
• Abstract summary: Refnd is a splitting algorithm that leverages a proximity graph computed in log time using Navigable Navi Small World (HNSW)<n>We validate on an antimicrobial peptide dataset, showing that Refnd splits yield lower but more realistic evaluation than traditional splits.
• Score: 3.5410731310203
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine learning models trained on biochemical data are routinely evaluated using splits that fail to account for relational structure, causing information leakage and over-optimistic performance estimates. Existing splitting methods lack theoretical grounding and scale at best quadratically. We introduce the Relational Generative Process (RGP), a mathematical formalization explaining why relational structure arises in biochemical datasets, and Refnd, a splitting algorithm that leverages a proximity graph computed in loglinear time using Hierarchical Navigable Small World (HNSW). We validate on an antimicrobial peptide dataset, showing that Refnd splits yield lower but more realistic evaluation performance than traditional splits. Refnd is applicable to any dataset arising from an RGP such as protein sequences and structures, small molecules, and nucleotide sequences, and is openly available as a Rust accelerated Python package: pip install refnd.

Related papers

• Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets [0.9003755151302326]
  Protein-protein interaction (PPI) databases do not faithfully reflect biological realities.<n>Machine learning models can exploit these as learning shortcuts if the negative dataset is not constructed with care.<n>Here, we systematically characterize both reported and, to our knowledge, previously unreported biases in PPI datasets that lead machine learning models to learn shortcuts instead of biological signal.
  arXiv  Detail & Related papers  (2026-09-09T14:02:18Z)
• REGEN: Reference-Guided Synthetic Multivariate Time Series Generation for Forecasting [7.771378647684901]
  We propose a reference-guided generative pipeline that treats observed sequences as structural scaffolds for synthesis.<n>We show that ReGeN-generated data consistently substitutes for real sibling data with minimal forecasting degradation.<n>This suggests that in low-data regimes, how reference data is structurally exploited can matter as much as how much data is available.
  arXiv  Detail & Related papers  (2026-06-03T16:19:55Z)
• Influence-Guided Symbolic Regression: Scientific Discovery via LLM-Driven Equation Search with Granular Feedback [56.69850045068714]
  We introduce textitInfluence-Guided Regression (IGSR), a method that frames equation discovery as an iterative two-step process.<n>We demonstrate IGSRs effectiveness on a diverse suite of benchmarks, including LLM-SRBench, pharmacological PKPD models, an epidemiological simulation, and real-world genomic data.
  arXiv  Detail & Related papers  (2026-05-27T23:48:01Z)
• Oxytrees: Model Trees for Bipartite Learning [1.3854111346209868]
  Oxytrees are proxy-based biclustering model trees that compress the interaction matrix into row- and column-wise proxy matrices.<n> Oxytrees employ linear models using the Kronecker product kernel in their leaves, resulting in shallower trees and thus even faster training.<n>We achieve up to 30-fold improvement in training times compared to state-of-the-art biclustering forests.
  arXiv  Detail & Related papers  (2025-11-16T17:57:13Z)
• Efficient Sparsification of Simplicial Complexes via Local Densities of States [18.733437653246543]
  Simplicial complexes (SCs) have become a popular abstraction for analyzing complex data.<n>We develop a novel method for a probabilistic sparsification of SCs that uses so-called local densities of states.<n>We demonstrate the performance of our framework on a family of Vietoris--Rips filtered simplicial complexes.
  arXiv  Detail & Related papers  (2025-02-11T13:51:42Z)
• Induced Covariance for Causal Discovery in Linear Sparse Structures [55.2480439325792]
  Causal models seek to unravel the cause-effect relationships among variables from observed data. This paper introduces a novel causal discovery algorithm designed for settings in which variables exhibit linearly sparse relationships.
  arXiv  Detail & Related papers  (2024-10-02T04:01:38Z)
• Minimally Supervised Learning using Topological Projections in Self-Organizing Maps [55.31182147885694]
  We introduce a semi-supervised learning approach based on topological projections in self-organizing maps (SOMs) Our proposed method first trains SOMs on unlabeled data and then a minimal number of available labeled data points are assigned to key best matching units (BMU) Our results indicate that the proposed minimally supervised model significantly outperforms traditional regression techniques.
  arXiv  Detail & Related papers  (2024-01-12T22:51:48Z)
• Harmless interpolation in regression and classification with structured features [21.064512161584872]
  Overparametrized neural networks tend to perfectly fit noisy training data yet generalize well on test data. We present a general and flexible framework for upper bounding regression and classification risk in a reproducing kernel Hilbert space.
  arXiv  Detail & Related papers  (2021-11-09T15:12:26Z)
• MIRACLE: Causally-Aware Imputation via Learning Missing Data Mechanisms [82.90843777097606]
  We propose a causally-aware imputation algorithm (MIRACLE) for missing data. MIRACLE iteratively refines the imputation of a baseline by simultaneously modeling the missingness generating mechanism. We conduct extensive experiments on synthetic and a variety of publicly available datasets to show that MIRACLE is able to consistently improve imputation.
  arXiv  Detail & Related papers  (2021-11-04T22:38:18Z)
• A Fast Parallel Tensor Decomposition with Optimal Stochastic Gradient Descent: an Application in Structural Damage Identification [1.536989504296526]
  We propose a novel algorithm, FP-CPD, to parallelize the CANDECOMP/PARAFAC (CP) decomposition of a tensor $mathcalX in mathbbR I_1 times dots times I_N $.
  arXiv  Detail & Related papers  (2021-11-04T05:17:07Z)
• Predictive Coding Approximates Backprop along Arbitrary Computation Graphs [68.8204255655161]
  We develop a strategy to translate core machine learning architectures into their predictive coding equivalents. Our models perform equivalently to backprop on challenging machine learning benchmarks. Our method raises the potential that standard machine learning algorithms could in principle be directly implemented in neural circuitry.
  arXiv  Detail & Related papers  (2020-06-07T15:35:47Z)
• New advances in enumerative biclustering algorithms with online partitioning [80.22629846165306]
  This paper further extends RIn-Close_CVC, a biclustering algorithm capable of performing an efficient, complete, correct and non-redundant enumeration of maximal biclusters with constant values on columns in numerical datasets. The improved algorithm is called RIn-Close_CVC3, keeps those attractive properties of RIn-Close_CVC, and is characterized by: a drastic reduction in memory usage; a consistent gain in runtime.
  arXiv  Detail & Related papers  (2020-03-07T14:54:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.