Tracing the Roots: A Multi-Agent Framework for Uncovering Data Lineage in Post-Training LLMs

• URL: http://arxiv.org/abs/2604.10480v1
• Date: Sun, 12 Apr 2026 06:24:07 GMT
• Title: Tracing the Roots: A Multi-Agent Framework for Uncovering Data Lineage in Post-Training LLMs
• Abstract summary: Post-training data plays a pivotal role in shaping the capabilities of Large Language Models (LLMs)<n>We propose an automated multi-agent framework to reconstruct the evolutionary graph of dataset development.
• Score: 81.43302841109349
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Post-training data plays a pivotal role in shaping the capabilities of Large Language Models (LLMs), yet datasets are often treated as isolated artifacts, overlooking the systemic connections that underlie their evolution. To disentangle these complex relationships, we introduce the concept of \textbf{data lineage} to the LLM ecosystem and propose an automated multi-agent framework to reconstruct the evolutionary graph of dataset development. Through large-scale lineage analysis, we characterize domain-specific structural patterns, such as vertical refinement in math-oriented datasets and horizontal aggregation in general-domain corpora. Moreover, we uncover pervasive systemic issues, including \textit{structural redundancy} induced by implicit dataset intersections and the \textit{propagation of benchmark contamination} along lineage paths. To demonstrate the practical value of lineage analysis for data construction, we leverage the reconstructed lineage graph to create a \textit{lineage-aware diversity-oriented dataset}. By anchoring instruction sampling at upstream root sources, this approach mitigates downstream homogenization and hidden redundancy, yielding a more diverse post-training corpus. We further highlight lineage-centric analysis as an efficient and robust topological alternative to sample-level dataset comparison for large-scale data ecosystems. By grounding data construction in explicit lineage structures, our work advances post-training data curation toward a more systematic and controllable paradigm.

Related papers

• Generative Data Transformation: From Mixed to Unified Data [57.84692191369066]
  textscTaesar is a emphdata-centric framework for textbftarget-textbfal textbfregeneration.<n>It encodes cross-domain context into target sequences, enabling standard models to learn intricate dependencies without complex fusion architectures.
  arXiv  Detail & Related papers  (2026-02-26T08:30:09Z)
• Beyond Human Annotation: Recent Advances in Data Generation Methods for Document Intelligence [6.0051533428647375]
  This survey establishes the first comprehensive technical map for data generation in Document Intelligence.<n>Data generation is redefined as supervisory signal production.<n>A novel taxonomy is introduced based on the "availability of data and labels"
  arXiv  Detail & Related papers  (2026-01-18T09:01:18Z)
• From Parameters to Performance: A Data-Driven Study on LLM Structure and Development [73.67759647072519]
  Large language models (LLMs) have achieved remarkable success across various domains.<n>Despite the rapid growth in model scale and capability, systematic, data-driven research on how structural configurations affect performance remains scarce.<n>We present a large-scale dataset encompassing diverse open-source LLM structures and their performance across multiple benchmarks.
  arXiv  Detail & Related papers  (2025-09-14T12:20:39Z)
• TaP: A Taxonomy-Guided Framework for Automated and Scalable Preference Data Generation [50.319535974012]
  Conducting supervised fine-tuning and preference fine-tuning on large language models (LLMs) requires high-quality datasets.<n>Most available datasets for supervised and preference fine-tuning are in English.<n>We propose the underlinetextbfTaxonomy-Guided underlinetextbfPreference Data Generation framework.
  arXiv  Detail & Related papers  (2025-06-30T15:45:28Z)
• Relational Deep Learning: Challenges, Foundations and Next-Generation Architectures [50.46688111973999]
  Graph machine learning has led to a significant increase in the capabilities of models that learn on arbitrary graph-structured data.<n>We present a new blueprint that enables end-to-end representation of'relational entity graphs' without traditional engineering feature.<n>We discuss key challenges including large-scale multi-table integration and the complexities of modeling temporal dynamics and heterogeneous data.
  arXiv  Detail & Related papers  (2025-06-19T23:51:38Z)
• Space of Data through the Lens of Multilevel Graph [0.0]
  This work seeks to tackle the inherent complexity of dataspaces by introducing a novel data structure.<n>We propose the concept of a multilevel graph, which is equipped with two fundamental operations: contraction and expansion of its topology.<n>We provide a comprehensive suite of methods for manipulating this graph structure, establishing a robust framework for data analysis.
  arXiv  Detail & Related papers  (2025-03-30T21:54:07Z)
• Topology-aware Reinforcement Feature Space Reconstruction for Graph Data [22.5530178427691]
  Reconstructing a good feature space is essential to augment the AI power of data, improve model generalization, and increase the availability of downstream ML models. We use topology-aware reinforcement learning to automate and optimize feature space reconstruction for graph data. Our approach combines the extraction of core subgraphs to capture essential structural information with a graph neural network (GNN) to encode topological features and reduce computing complexity.
  arXiv  Detail & Related papers  (2024-11-08T18:01:05Z)
• Entropic Optimal Transport Eigenmaps for Nonlinear Alignment and Joint Embedding of High-Dimensional Datasets [11.105392318582677]
  We propose a principled approach for aligning and jointly embedding a pair of datasets with theoretical guarantees. Our approach leverages the leading singular vectors of the EOT plan matrix between two datasets to extract their shared underlying structure. We show that in a high-dimensional regime, the EOT plan recovers the shared manifold structure by approximating a kernel function evaluated at the locations of the latent variables.
  arXiv  Detail & Related papers  (2024-07-01T18:48:55Z)
• On Linearizing Structured Data in Encoder-Decoder Language Models: Insights from Text-to-SQL [8.57550491437633]
  This work investigates the linear handling of structured data in encoder-decoder language models, specifically T5. Our findings reveal the model's ability to mimic human-designed processes such as schema linking and syntax prediction. We also uncover insights into the model's internal mechanisms, including the ego-centric nature of structure node encodings.
  arXiv  Detail & Related papers  (2024-04-03T01:16:20Z)
• AgentOhana: Design Unified Data and Training Pipeline for Effective Agent Learning [98.26836657967162]
  textbfAgentOhana aggregates agent trajectories from distinct environments, spanning a wide array of scenarios. textbfxLAM-v0.1, a large action model tailored for AI agents, demonstrates exceptional performance across various benchmarks.
  arXiv  Detail & Related papers  (2024-02-23T18:56:26Z)
• Joint Geometric and Topological Analysis of Hierarchical Datasets [7.098759778181621]
  In this paper, we focus on high-dimensional data that are organized into several hierarchical datasets. The main novelty in this work lies in the combination of two powerful data-analytic approaches: topological data analysis and geometric manifold learning. We show that our new method gives rise to superior classification results compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-04-03T13:02:00Z)
• Meta-learning framework with applications to zero-shot time-series forecasting [82.61728230984099]
  This work provides positive evidence using a broad meta-learning framework. residual connections act as a meta-learning adaptation mechanism. We show that it is viable to train a neural network on a source TS dataset and deploy it on a different target TS dataset without retraining.
  arXiv  Detail & Related papers  (2020-02-07T16:39:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.