Data Reconstruction: Identifiability and Optimization with Sample Splitting

• URL: http://arxiv.org/abs/2602.08723v1
• Date: Mon, 09 Feb 2026 14:33:19 GMT
• Title: Data Reconstruction: Identifiability and Optimization with Sample Splitting
• Abstract summary: Training data reconstruction from KKT conditions has shown striking empirical success.<n>It remains unclear when the resulting KKT equations have unique solutions and, even in identifiable regimes, how to reliably recover solutions by optimization.
• Score: 28.085421010299758
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Training data reconstruction from KKT conditions has shown striking empirical success, yet it remains unclear when the resulting KKT equations have unique solutions and, even in identifiable regimes, how to reliably recover solutions by optimization. This work hereby focuses on these two complementary questions: identifiability and optimization. On the identifiability side, we discuss the sufficient conditions for KKT system of two-layer networks with polynomial activations to uniquely determine the training data, providing a theoretical explanation of when and why reconstruction is possible. On the optimization side, we introduce sample splitting, a curvature-aware refinement step applicable to general reconstruction objectives (not limited to KKT-based formulations): it creates additional descent directions to escape poor stationary points and refine solutions. Experiments demonstrate that augmenting several existing reconstruction methods with sample splitting consistently improves reconstruction performance.

Related papers

• CaIRec: Calibrated Modality Imputation for Incomplete Multimodal Recommendation [49.24608818070328]
  Real-world recommender systems often face incomplete modality observations, where items lack images, text, or other content features.<n>Existing modality imputation methods estimate missing representations from available item content, but two challenges remain.<n>First, they optimize the recovered representation itself without explicitly considering its relations with other modalities of the same item.<n>Second, even structurally coherent recovered information may remain ineffective for personalized ranking.<n>We propose Calibrated Imputation for Incomplete Multimodal Recommendation (CaIRec) to address these challenges.
  arXiv  Detail & Related papers  (2026-07-29T10:10:20Z)
• Adaptive Latent Trajectory Anchoring for Action Segmentation Dataset Condensation [53.60856261088647]
  This paper proposes to shift the condensation paradigm from optimization-based inversion to deterministic latent mapping.<n>We represent action segments as continuous trajectories anchored by sparse latent points in the noise manifold.<n>Our framework significantly outperforms state-of-the-art methods in segmentation performance across common datasets.
  arXiv  Detail & Related papers  (2026-07-10T04:00:26Z)
• Adaptive Recurrent Message Passing for Test Time Computing on Graphs [80.56829418752513]
  We show that recurrent graph models can overcome the intrinsic mismatch between graph data and architectural designs.<n>We propose AdaR, an Adaptive Recurrent graph model, empowering flexible test-time computing on various downstream tasks without changing model parameters.
  arXiv  Detail & Related papers  (2026-06-21T12:14:11Z)
• Structure-Aware Robust Counterfactual Explanations via Conditional Gaussian Network Classifiers [0.26999000177990923]
  This work presents a structure-aware robustness-and-counterfactual search method based on conditional conditional graphs.<n>Results show that our method achieves strong consistency, with direct optimization of the original formulation providing especially stable dependencies.<n>The proposed framework lays the groundwork for future advances in counterfactual reasoning under noncyclic constraints.
  arXiv  Detail & Related papers  (2026-02-08T15:51:45Z)
• CoT Referring: Improving Referring Expression Tasks with Grounded Reasoning [67.18702329644526]
  CoT Referring enhances model reasoning across modalities through a structured, chain-of-thought training data structure.<n>We restructure the training data to enforce a new output form, providing new annotations for existing datasets.<n>We also integrate detection and segmentation capabilities into a unified MLLM framework, training it with a novel adaptive weighted loss to optimize performance.
  arXiv  Detail & Related papers  (2025-10-03T08:50:21Z)
• Flow-Matching Guided Deep Unfolding for Hyperspectral Image Reconstruction [53.26903617819014]
  Flow-Matching-guided Unfolding network (FMU) is first to integrate flow matching into HSI reconstruction.<n>To further strengthen the learned dynamics, we introduce a mean velocity loss.<n>Experiments on both simulated and real datasets show that FMU significantly outperforms existing approaches in reconstruction quality.
  arXiv  Detail & Related papers  (2025-10-02T11:32:00Z)
• How Far Are We from Generating Missing Modalities with Foundation Models? [49.425856207329524]
  We propose an agentic framework tailored for missing modality reconstruction.<n>Our method reduces FID for missing image reconstruction by at least 14% and MER for missing text reconstruction by at least 10% compared to baselines.
  arXiv  Detail & Related papers  (2025-06-04T03:22:44Z)
• Constrained Bayesian Optimization Under Partial Observations: Balanced Improvements and Provable Convergence [6.461785985849886]
  We endeavor to design an efficient and provable method for expensive POCOPs under the framework of constrained Bayesian optimization. We present an improved design of the acquisition functions that introduces balanced exploration during optimization. We propose a Gaussian process embedding different likelihoods as the surrogate model for a partially observable constraint.
  arXiv  Detail & Related papers  (2023-12-06T01:00:07Z)
• Stable Nonconvex-Nonconcave Training via Linear Interpolation [51.668052890249726]
  This paper presents a theoretical analysis of linearahead as a principled method for stabilizing (large-scale) neural network training. We argue that instabilities in the optimization process are often caused by the nonmonotonicity of the loss landscape and show how linear can help by leveraging the theory of nonexpansive operators.
  arXiv  Detail & Related papers  (2023-10-20T12:45:12Z)
• Optimal Sets and Solution Paths of ReLU Networks [56.40911684005949]
  We develop an analytical framework to characterize the set of optimal ReLU networks. We establish conditions for the neuralization of ReLU networks to be continuous, and develop sensitivity results for ReLU networks.
  arXiv  Detail & Related papers  (2023-05-31T18:48:16Z)
• Benign Overfitting in Linear Classifiers and Leaky ReLU Networks from KKT Conditions for Margin Maximization [59.038366742773164]
  Linears and leaky ReLU trained by gradient flow on logistic loss have an implicit bias towards satisfying the Karush-KuTucker (KKT) conditions. In this work we establish a number of settings where the satisfaction of these conditions implies benign overfitting in linear classifiers and in two-layer leaky ReLU networks.
  arXiv  Detail & Related papers  (2023-03-02T18:24:26Z)
• Backpropagation of Unrolled Solvers with Folded Optimization [55.04219793298687]
  The integration of constrained optimization models as components in deep networks has led to promising advances on many specialized learning tasks. One typical strategy is algorithm unrolling, which relies on automatic differentiation through the operations of an iterative solver. This paper provides theoretical insights into the backward pass of unrolled optimization, leading to a system for generating efficiently solvable analytical models of backpropagation.
  arXiv  Detail & Related papers  (2023-01-28T01:50:42Z)
• Causal Structural Learning from Time Series: A Convex Optimization Approach [12.4517307615083]
  Structural learning aims to learn directed acyclic graphs (DAGs) from observational data. Recent DAG learning remains a highly non-adaptive structural learning problem. We propose a data approach for causal learning using a recently developed monotone variational (VI) formulation.
  arXiv  Detail & Related papers  (2023-01-26T16:39:58Z)
• Rethinking the optimization process for self-supervised model-driven MRI reconstruction [16.5013498806588]
  K2Calibrate is a K-space adaptation strategy for self-supervised model-driven MR reconstruction optimization. It can reduce the network's reconstruction deterioration caused by statistically dependent noise. It achieves better results than five state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-03-18T03:41:36Z)
• Self-Supervised Learning for MRI Reconstruction with a Parallel Network Training Framework [24.46388892324129]
  The proposed method is flexible and can be employed in any existing deep learning-based method. The effectiveness of the method is evaluated on an open brain MRI dataset.
  arXiv  Detail & Related papers  (2021-09-26T06:09:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.