Efficient Techniques for Data Reconstruction, with Finite-Width Recovery Guarantees

• URL: http://arxiv.org/abs/2605.06519v1
• Date: Thu, 07 May 2026 16:27:12 GMT
• Title: Efficient Techniques for Data Reconstruction, with Finite-Width Recovery Guarantees
• Abstract summary: Data reconstruction attacks on trained neural networks aim to recover the data on which the network has been trained.<n>We propose a unified optimization formulation of the data reconstruction problem based on initial and trained parameter values.<n>We show that in the random feature model, this formulation provably leads to training data reconstruction with high probability.
• Score: 0.11666234644810893
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Data reconstruction attacks on trained neural networks aim to recover the data on which the network has been trained and pose a significant threat to privacy, especially if the training dataset contains sensitive information. Here, we propose a unified optimization formulation of the data reconstruction problem based on initial and trained parameter values, incorporating state-of-the-art proposals. We show that in the random feature model, this formulation provably leads to training data reconstruction with high probability, provided the network width is sufficiently large; this unprecedented finite-width result uses PAC-style bounds. Furthermore, when the data lies in a low-dimensional subspace, we show that the network width requirement for successful reconstruction can be relaxed, with bounds depending on the subspace dimension rather than the ambient dimension. For general neural network models and unknown data orientations, we propose an efficient reconstruction algorithm that approximates the low-dimensional data subspace through the change in the first-layer weights during training and uses only the last-layer weights for reconstruction, thus reducing the search space dimension and the required network width for high-quality reconstructions. Our numerical experiments on synthetic datasets and CIFAR-10 confirm that our subspace-aware reconstruction approach outperforms standard full-space techniques.

Related papers

• Revisiting the Privacy Risks of Split Inference: A GAN-Based Data Reconstruction Attack via Progressive Feature Optimization [49.32786615205064]
  Split Inference (SI) partitions computation between edge devices and the cloud to reduce latency and protect user privacy.<n>Recent advances in Data Reconstruction Attacks (DRAs) reveal that intermediate features exchanged in SI can be exploited to recover sensitive input data.<n>Existing DRAs are typically effective only on shallow models and fail to fully leverage semantic priors.<n>We propose a novel GAN-based DRA framework with Progressive Feature Optimization (PFO), which decomposes the generator into hierarchical blocks and incrementally refines intermediate representations to enhance the semantic fidelity of reconstructed images.
  arXiv  Detail & Related papers  (2025-08-28T10:00:39Z)
• Dual-branch Graph Feature Learning for NLOS Imaging [51.31554007495926]
  Non-line-of-sight (NLOS) imaging offers the capability to reveal occluded scenes that are not directly visible.<n>xnet methodology integrates an albedo-focused reconstruction branch dedicated to albedo information recovery and a depth-focused reconstruction branch that extracts geometrical structure.<n>Our method attains the highest level of performance among existing methods across synthetic and real data.
  arXiv  Detail & Related papers  (2025-02-27T01:49:00Z)
• Re-Visible Dual-Domain Self-Supervised Deep Unfolding Network for MRI Reconstruction [48.30341580103962]
  We propose a novel re-visible dual-domain self-supervised deep unfolding network to address these issues.<n>We design a deep unfolding network based on Chambolle and Pock Proximal Point Algorithm (DUN-CP-PPA) to achieve end-to-end reconstruction.<n> Experiments conducted on the fastMRI and IXI datasets demonstrate that our method significantly outperforms state-of-the-art approaches in terms of reconstruction performance.
  arXiv  Detail & Related papers  (2025-01-07T12:29:32Z)
• SeisFusion: Constrained Diffusion Model with Input Guidance for 3D Seismic Data Interpolation and Reconstruction [26.02191880837226]
  We propose a novel diffusion model reconstruction framework tailored for 3D seismic data. We introduce a 3D neural network architecture into the diffusion model, successfully extending the 2D diffusion model to 3D space. Our method exhibits superior reconstruction accuracy when applied to both field datasets and synthetic datasets.
  arXiv  Detail & Related papers  (2024-03-18T05:10:13Z)
• SST-ReversibleNet: Reversible-prior-based Spectral-Spatial Transformer for Efficient Hyperspectral Image Reconstruction [15.233185887461826]
  A novel framework called the reversible-prior-based method is proposed. ReversibleNet significantly outperforms state-of-the-art methods on simulated and real HSI datasets.
  arXiv  Detail & Related papers  (2023-05-06T14:01:02Z)
• Understanding Reconstruction Attacks with the Neural Tangent Kernel and Dataset Distillation [110.61853418925219]
  We build a stronger version of the dataset reconstruction attack and show how it can provably recover the emphentire training set in the infinite width regime. We show that both theoretically and empirically, reconstructed images tend to "outliers" in the dataset. These reconstruction attacks can be used for textitdataset distillation, that is, we can retrain on reconstructed images and obtain high predictive accuracy.
  arXiv  Detail & Related papers  (2023-02-02T21:41:59Z)
• Bayesian Interpolation with Deep Linear Networks [92.1721532941863]
  Characterizing how neural network depth, width, and dataset size jointly impact model quality is a central problem in deep learning theory. We show that linear networks make provably optimal predictions at infinite depth. We also show that with data-agnostic priors, Bayesian model evidence in wide linear networks is maximized at infinite depth.
  arXiv  Detail & Related papers  (2022-12-29T20:57:46Z)
• Deep Manifold Prior [37.725563645899584]
  We present a prior for manifold structured data, such as surfaces of 3D shapes, where deep neural networks are adopted to reconstruct a target shape using gradient descent. We show that surfaces generated this way are smooth, with limiting behavior characterized by Gaussian processes, and we mathematically derive such properties for fully-connected as well as convolutional networks.
  arXiv  Detail & Related papers  (2020-04-08T20:47:56Z)
• Large-Scale Gradient-Free Deep Learning with Recursive Local Representation Alignment [84.57874289554839]
  Training deep neural networks on large-scale datasets requires significant hardware resources. Backpropagation, the workhorse for training these networks, is an inherently sequential process that is difficult to parallelize. We propose a neuro-biologically-plausible alternative to backprop that can be used to train deep networks.
  arXiv  Detail & Related papers  (2020-02-10T16:20:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.