Knowledge-Embedded Latent Projection for Robust Representation Learning

• URL: http://arxiv.org/abs/2602.16709v1
• Date: Wed, 18 Feb 2026 18:58:16 GMT
• Title: Knowledge-Embedded Latent Projection for Robust Representation Learning
• Authors: Weijing Tang, Ming Yuan, Zongqi Xia, Tianxi Cai,
• Abstract summary: We propose a knowledge-embedded latent projection model that leverages semantic side information to regularize representation learning.<n>Specifically, we model column embeddings as smooth functions of semantic embeddings via a kernel mapping in a Hilbert space.<n>We develop a computationally efficient two-step embedding estimation procedure that combines semantically guided subspace construction via kernel principal component embedding.
• Score: 5.79422287722755
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Latent space models are widely used for analyzing high-dimensional discrete data matrices, such as patient-feature matrices in electronic health records (EHRs), by capturing complex dependence structures through low-dimensional embeddings. However, estimation becomes challenging in the imbalanced regime, where one matrix dimension is much larger than the other. In EHR applications, cohort sizes are often limited by disease prevalence or data availability, whereas the feature space remains extremely large due to the breadth of medical coding system. Motivated by the increasing availability of external semantic embeddings, such as pre-trained embeddings of clinical concepts in EHRs, we propose a knowledge-embedded latent projection model that leverages semantic side information to regularize representation learning. Specifically, we model column embeddings as smooth functions of semantic embeddings via a mapping in a reproducing kernel Hilbert space. We develop a computationally efficient two-step estimation procedure that combines semantically guided subspace construction via kernel principal component analysis with scalable projected gradient descent. We establish estimation error bounds that characterize the trade-off between statistical error and approximation error induced by the kernel projection. Furthermore, we provide local convergence guarantees for our non-convex optimization procedure. Extensive simulation studies and a real-world EHR application demonstrate the effectiveness of the proposed method.

Related papers

• SIGMA: Scalable Spectral Insights for LLM Collapse [51.863164847253366]
  We introduce SIGMA (Spectral Inequalities for Gram Matrix Analysis), a unified framework for model collapse.<n>By utilizing benchmarks that deriving and deterministic bounds on the matrix's spectrum, SIGMA provides a mathematically grounded metric to track the contraction of the representation space.<n>We demonstrate that SIGMA effectively captures the transition towards states, offering both theoretical insights into the mechanics of collapse.
  arXiv  Detail & Related papers  (2026-01-06T19:47:11Z)
• Disentangling Interpretable Factors with Supervised Independent Subspace Principal Component Analysis [0.9558392439655012]
  Supervised Independent Subspace Principal Component Analysis ($texttsisPCA$) is a PCA extension designed for multi-subspace learning. We demonstrate its ability to identify and separate hidden data structures through extensive applications, including breast cancer diagnosis. Our results reveal distinct functional pathways associated with malaria colonization, underscoring the essentiality of explainable representation in high-dimensional data analysis.
  arXiv  Detail & Related papers  (2024-10-31T03:09:40Z)
• Efficient High-Resolution Visual Representation Learning with State Space Model for Human Pose Estimation [60.80423207808076]
  Capturing long-range dependencies while preserving high-resolution visual representations is crucial for dense prediction tasks such as human pose estimation.<n>We propose the Dynamic Visual State Space (DVSS) block, which augments visual state space models with multi-scale convolutional operations.<n>We build HRVMamba, a novel model for efficient high-resolution representation learning.
  arXiv  Detail & Related papers  (2024-10-04T06:19:29Z)
• Seeing Unseen: Discover Novel Biomedical Concepts via Geometry-Constrained Probabilistic Modeling [53.7117640028211]
  We present a geometry-constrained probabilistic modeling treatment to resolve the identified issues. We incorporate a suite of critical geometric properties to impose proper constraints on the layout of constructed embedding space. A spectral graph-theoretic method is devised to estimate the number of potential novel classes.
  arXiv  Detail & Related papers  (2024-03-02T00:56:05Z)
• Synergistic eigenanalysis of covariance and Hessian matrices for enhanced binary classification [72.77513633290056]
  We present a novel approach that combines the eigenanalysis of a covariance matrix evaluated on a training set with a Hessian matrix evaluated on a deep learning model. Our method captures intricate patterns and relationships, enhancing classification performance.
  arXiv  Detail & Related papers  (2024-02-14T16:10:42Z)
• Inference of Dependency Knowledge Graph for Electronic Health Records [15.711857066677956]
  We propose a framework for deriving a sparse knowledge graph based on the dynamic log-linear topic model.<n>Within this model, the KG embeddings are estimated by performing singular value decomposition on the empirical pointwise mutual information matrix.<n>We then establish entrywise normality for the KG low-rank estimator, enabling the recovery of sparse graph edges with controlled type I error.
  arXiv  Detail & Related papers  (2023-12-25T04:45:36Z)
• Directed Cyclic Graph for Causal Discovery from Multivariate Functional Data [15.26007975367927]
  We introduce a functional linear structural equation model for causal structure learning. To enhance interpretability, our model involves a low-dimensional causal embedded space. We prove that the proposed model is causally identifiable under standard assumptions.
  arXiv  Detail & Related papers  (2023-10-31T15:19:24Z)
• Understanding Augmentation-based Self-Supervised Representation Learning via RKHS Approximation and Regression [53.15502562048627]
  Recent work has built the connection between self-supervised learning and the approximation of the top eigenspace of a graph Laplacian operator. This work delves into a statistical analysis of augmentation-based pretraining.
  arXiv  Detail & Related papers  (2023-06-01T15:18:55Z)
• Statistical Inference for Linear Functionals of Online SGD in High-dimensional Linear Regression [7.884611719110979]
  gradient descent (SGD) has emerged as the quintessential method in a data scientist's toolbox.<n>We establish a high-dimensional Central Limit Theorem (CLT) for linear functionals of online SGD iterates for overparametrized least-squares regression with non-isotropic Gaussian inputs.<n>We develop an online approach for estimating the variance term appearing in the CLT, and establish high-probability bounds for the developed online estimator.
  arXiv  Detail & Related papers  (2023-02-20T02:38:36Z)
• Validation Diagnostics for SBI algorithms based on Normalizing Flows [55.41644538483948]
  This work proposes easy to interpret validation diagnostics for multi-dimensional conditional (posterior) density estimators based on NF. It also offers theoretical guarantees based on results of local consistency. This work should help the design of better specified models or drive the development of novel SBI-algorithms.
  arXiv  Detail & Related papers  (2022-11-17T15:48:06Z)
• Deep Efficient Continuous Manifold Learning for Time Series Modeling [11.876985348588477]
  A symmetric positive definite matrix is being studied in computer vision, signal processing, and medical image analysis. In this paper, we propose a framework to exploit a diffeomorphism mapping between Riemannian manifold and a Cholesky space. For dynamic modeling of time-series data, we devise a continuous manifold learning method by systematically integrating a manifold ordinary differential equation and a gated recurrent neural network.
  arXiv  Detail & Related papers  (2021-12-03T01:38:38Z)
• Data-driven Uncertainty Quantification in Computational Human Head Models [0.6745502291821954]
  Modern biofidelic head model simulations are associated with very high computational cost and high-dimensional inputs and outputs. In this study, a two-stage, data-driven manifold learning-based framework is proposed for uncertainty quantification (UQ) of computational head models. It is demonstrated that the surrogate models provide highly accurate approximations of the computational model while significantly reducing the computational cost.
  arXiv  Detail & Related papers  (2021-10-29T05:42:31Z)
• Statistical control for spatio-temporal MEG/EEG source imaging with desparsified multi-task Lasso [102.84915019938413]
  Non-invasive techniques like magnetoencephalography (MEG) or electroencephalography (EEG) offer promise of non-invasive techniques. The problem of source localization, or source imaging, poses however a high-dimensional statistical inference challenge. We propose an ensemble of desparsified multi-task Lasso (ecd-MTLasso) to deal with this problem.
  arXiv  Detail & Related papers  (2020-09-29T21:17:16Z)
• High-dimensional Bayesian Optimization of Personalized Cardiac Model Parameters via an Embedded Generative Model [7.286540513944084]
  We present a novel concept that embeds a generative variational auto-encoder (VAE) into the objective function of Bayesian optimization. VAE-encoded knowledge about the generative code is used to guide the exploration of the search space. The presented method is applied to estimating tissue excitability in a cardiac electrophysiological model.
  arXiv  Detail & Related papers  (2020-05-15T22:14:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.