Towards a Data-Parameter Correspondence for LLMs: A Preliminary Discussion

• URL: http://arxiv.org/abs/2604.17384v1
• Date: Sun, 19 Apr 2026 11:18:03 GMT
• Title: Towards a Data-Parameter Correspondence for LLMs: A Preliminary Discussion
• Abstract summary: Large language model optimization has historically bifurcated into isolated data-centric and model-centric paradigms.<n>This paper establishes a unified emphdata- parameter correspondence, revealing these seemingly disparate operations as dual manifestations of the same geometric structure.
• Score: 10.237445112597095
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language model optimization has historically bifurcated into isolated data-centric and model-centric paradigms: the former manipulates involved samples through selection, augmentation, or poisoning, while the latter tunes model weights via masking, quantization, or low-rank adaptation. This paper establishes a unified \emph{data-parameter correspondence} revealing these seemingly disparate operations as dual manifestations of the same geometric structure on the statistical manifold $\mathcal{M}$. Grounded in the Fisher-Rao metric $g_{ij}(θ)$ and Legendre duality between natural ($θ$) and expectation ($η$) parameters, we identify three fundamental correspondences spanning the model lifecycle: 1. Geometric correspondence: data pruning and parameter sparsification equivalently reduce manifold volume via dual coordinate constraints; 2. Low-rank correspondence: in-context learning (ICL) and LoRA adaptation explore identical subspaces on the Grassmannian $\mathcal{G}(r,d)$, with $k$-shot samples geometrically equivalent to rank-$r$ updates; 3. Security-privacy correspondence: adversarial attacks exhibit cooperative amplification between data poisoning and parameter backdoors, whereas protective mechanisms follow cascading attenuation where data compression multiplicatively enhances parameter privacy. Extending from training through post-training compression to inference, this framework provides mathematical formalization for cross-community methodology transfer, demonstrating that cooperative optimization integrating data and parameter modalities may outperform isolated approaches across efficiency, robustness, and privacy dimensions.

Related papers

• Generative Criticality in Large Language Model Temperature Scaling [3.472734054649125]
  We propose a statistical-field framework for text generated by large language models (LLMs)<n>We define a susceptibility from the connected two-point correlator and an order parameter from the ensemble-averaged embedding field.<n>We observe a sharp susceptibility peak near a characteristic $T_c$ with power-law-like scaling, a concurrent rapid change in the order parameter, and a collapse onto a single semantic direction below $T_c$.
  arXiv  Detail & Related papers  (2026-06-04T14:43:32Z)
• Machine Learning Surrogate Modeling for Homogenization of Hyperelastic Materials with Boolean Microstructures [0.0]
  Data-driven surrogate models are an alternative to numerical homogenization of heterogeneous materials.<n>A neural network is trained on combinations of scalar and curve-wise statistical descriptors.<n>Additional data representing limiting cases of the parameter space are incorporated to stabilize training and improve extrapolation behavior.
  arXiv  Detail & Related papers  (2026-05-31T00:51:52Z)
• Joint Model and Data Sparsification via the Marginal Likelihood [53.29070892356214]
  We propose jointly learning individual feature and sample relevancies, enabling simultaneous model and data sparsification.<n>This symmetric pruning of model and data offers a natural extension that preserves conjugacy.<n> Empirical results across diverse regression tasks affirm that a joint ARD approach consistently yields both sparse and robust prediction models.
  arXiv  Detail & Related papers  (2026-05-28T13:26:53Z)
• A Bayesian Framework for Symmetry Inference in Chaotic Attractors [0.0]
  We present a framework that formulates symmetry detection as probabilistic model selection over a lattice of candidate subgroups.<n>An application to human gait dynamics reveals symmetry changes induced by mechanical constraints.
  arXiv  Detail & Related papers  (2025-10-18T13:49:35Z)
• Dendrograms of Mixing Measures for Softmax-Gated Gaussian Mixture of Experts: Consistency without Model Sweeps [41.371172458797524]
  Non-identifiability of gating parameters up to common translations, intrinsic gate-expert interactions, and tight numerator-denominator coupling are addressed.<n>For model selection, we adapt dendrogram-guided SGMoE, yielding a consistent, sweep-free selector of the number of experts that attains optimal parameter rates.<n>On a dataset of drought-identifiable maize traits, our dendrogram-guided SGMoE selects two experts, exposes a clear mixing hierarchy, stabilizes the likelihood early, and yields interpretable genotype-phenotype maps.
  arXiv  Detail & Related papers  (2025-10-14T17:23:44Z)
• Communication-Efficient and Personalized Federated Foundation Model Fine-Tuning via Tri-Matrix Adaptation [47.82423317739088]
  This paper introduces communication-efficient federated LoRA adaption (CE-LoRA), a method that employs a tri-factorization low-rank adaptation approach with personalized model parameter aggregation.<n>Experiments on various LLM and VLM fine-tuning tasks demonstrate that CE-LoRA not only significantly reduces communication overhead but also improves performance under not independently and identically distributed data conditions.
  arXiv  Detail & Related papers  (2025-03-31T09:18:42Z)
• ALoRE: Efficient Visual Adaptation via Aggregating Low Rank Experts [71.91042186338163]
  ALoRE is a novel PETL method that reuses the hypercomplex parameterized space constructed by Kronecker product to Aggregate Low Rank Experts.<n>Thanks to the artful design, ALoRE maintains negligible extra parameters and can be effortlessly merged into the frozen backbone.
  arXiv  Detail & Related papers  (2024-12-11T12:31:30Z)
• Latent Semantic Consensus For Deterministic Geometric Model Fitting [109.44565542031384]
  We propose an effective method called Latent Semantic Consensus (LSC) LSC formulates the model fitting problem into two latent semantic spaces based on data points and model hypotheses. LSC is able to provide consistent and reliable solutions within only a few milliseconds for general multi-structural model fitting.
  arXiv  Detail & Related papers  (2024-03-11T05:35:38Z)
• Data-free Weight Compress and Denoise for Large Language Models [96.68582094536032]
  We propose a novel approach termed Data-free Joint Rank-k Approximation for compressing the parameter matrices.<n>We achieve a model pruning of 80% parameters while retaining 93.43% of the original performance without any calibration data.
  arXiv  Detail & Related papers  (2024-02-26T05:51:47Z)
• Heterogeneous Multi-Task Gaussian Cox Processes [61.67344039414193]
  We present a novel extension of multi-task Gaussian Cox processes for modeling heterogeneous correlated tasks jointly. A MOGP prior over the parameters of the dedicated likelihoods for classification, regression and point process tasks can facilitate sharing of information between heterogeneous tasks. We derive a mean-field approximation to realize closed-form iterative updates for estimating model parameters.
  arXiv  Detail & Related papers  (2023-08-29T15:01:01Z)
• Surrogate Assisted Strategies (The Parameterisation of an Infectious Disease Agent-Based Model) [0.0]
  calibration is a significant challenge in agent-based modelling and simulation. We propose a more comprehensive and adaptive ABMS Framework that can effectively swap out parameterisation strategies and surrogate models. We show in a real-world setting that DYCORS XGBoost and MSRS SVM can approximate the real world cumulative daily infection distribution.
  arXiv  Detail & Related papers  (2021-08-19T17:27:01Z)
• MINIMALIST: Mutual INformatIon Maximization for Amortized Likelihood Inference from Sampled Trajectories [61.3299263929289]
  Simulation-based inference enables learning the parameters of a model even when its likelihood cannot be computed in practice. One class of methods uses data simulated with different parameters to infer an amortized estimator for the likelihood-to-evidence ratio. We show that this approach can be formulated in terms of mutual information between model parameters and simulated data.
  arXiv  Detail & Related papers  (2021-06-03T12:59:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.