Mechanistic Circuit Identification for Controllable Data Generation

• URL: http://arxiv.org/abs/2608.24065v1
• Date: Tue, 25 Aug 2026 04:53:19 GMT
• Title: Mechanistic Circuit Identification for Controllable Data Generation
• Abstract summary: We propose a circuit-grounded framework that connects training-dynamics-based data valuation with mechanistic interpretability.<n> Specifically, we conceptualize data quality along three complementary utility axes, learnability, challenge, and alignment.<n>Building on this capability, we introduce SAMS (Stage-Aware Mechanistic Scheduling), which schedules circuit-steered data according to the model's evolving optimization needs.
• Score: 8.346479730119436
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While recent advances in data synthesis aim to curate high-quality datasets, most generation pipelines still rely on heuristic prompt-based control. This black-box paradigm provides limited insight into how individual samples interact with a model's underlying learning dynamics. To bridge this gap, we propose a circuit-grounded framework that connects training-dynamics-based data valuation with mechanistic interpretability (MI). Specifically, we conceptualize data quality along three complementary utility axes, learnability, challenge, and alignment. First, we uncover specialized model-internal circuits that causally govern these utility signals. Then, moving beyond heuristic prompting toward mechanistic control, we leverage these circuits as controllable interfaces, actively steering generation to produce utility-targeted data. Building on this capability, we introduce SAMS (Stage-Aware Mechanistic Scheduling), which schedules circuit-steered data according to the model's evolving optimization needs. Experiments on multiple-choice QA tasks demonstrate that our approach yields precisely controlled data with greater diversity than prompt-based baselines, consistently improving downstream performance and calibration. Ultimately, this work establishes a principled white-box paradigm for interpretable data generation, pioneering the use of MI not just as an analytical tool, but as a practical, controllable interface.

Related papers

• Modular Foundation Models for Time-Series Perception in Digital Twins [0.8983000676504388]
  Engineering Digital Twins and Prognostics and Health Management (PHM) systems rely on robust perception modules to extract actionable information from time-series data.<n>This paper proposes a modular foundation model for time-series perception based on a collection of pretrained representation encoders.<n>The framework leverages self-supervised learning on heterogeneous datasets to learn transferable and task-agnostic representations, which can be reused across multiple PHM tasks.
  arXiv  Detail & Related papers  (2026-07-03T20:02:28Z)
• Learning Multi-Modal Trajectory Policies for Data-Efficient Robotic Manipulation [21.99077052779724]
  MATE (Multi-Modal MoE) is a novel trajectory prediction framework built upon MoE.<n>We introduce a Multi-Modal MoE architecture to achieve fine-grained sub-modal cosine, and design a cross-modal router for stable and scale-invariant expert assignment.<n>Experiments on robotic ping-pong show that our MATE consistently outperforms prior work under data scarcity.
  arXiv  Detail & Related papers  (2026-05-31T06:35:10Z)
• GO-GenZip: Goal-Oriented Generative Sampling and Hybrid Compression [50.76941829813458]
  Current network data telemetry pipelines consist of massive streams of fine-grained Key Performance Indicators (KPIs) from multiple distributed sources towards central aggregators.<n>This work presents a generative AI (GenAI)-driven sampling and hybrid compression framework that redesigns network telemetry from a goal-oriented perspective.
  arXiv  Detail & Related papers  (2026-03-20T16:33:15Z)
• Data-Driven Control of a Magnetically Actuated Fish-Like Robot [2.5782420501870296]
  Magnetically actuated fish-like robots offer promising solutions for underwater exploration due to their miniaturization and agility.<n>This paper proposes a data-driven control framework to address these complexities without relying on analytical modeling.
  arXiv  Detail & Related papers  (2026-03-05T04:08:08Z)
• Multi-Task Equation Discovery [0.0]
  We use a multi-task learning framework for simultaneous parameter identification across multiple datasets.<n>The MTL-RVM combined information across tasks, improving parameter recovery for weakly and moderately excited datasets.<n>These findings demonstrate that multi-task Bayesian inference can mitigate over-fitting and promote generalisation in equation discovery.
  arXiv  Detail & Related papers  (2025-09-29T18:56:40Z)
• Neural Network Reprogrammability: A Unified Theme on Model Reprogramming, Prompt Tuning, and Prompt Instruction [57.19302613163439]
  We introduce neural network reprogrammability as a unifying framework for model adaptation.<n>We present a taxonomy that categorizes such information manipulation approaches across four key dimensions.<n>We also analyze remaining technical challenges and ethical considerations.
  arXiv  Detail & Related papers  (2025-06-05T05:42:27Z)
• Generative QoE Modeling: A Lightweight Approach for Telecom Networks [6.473372512447993]
  This study introduces a lightweight generative modeling framework that balances computational efficiency, interpretability, and predictive accuracy.<n>By validating the use of Vector Quantization (VQ) as a preprocessing technique, continuous network features are effectively transformed into discrete categorical symbols.<n>This VQ-HMM pipeline enhances the model's capacity to capture dynamic QoE patterns while supporting probabilistic inference on new and unseen data.
  arXiv  Detail & Related papers  (2025-04-30T06:19:37Z)
• Data Scaling Laws for End-to-End Autonomous Driving [83.85463296830743]
  We evaluate the performance of a simple end-to-end driving architecture on internal driving datasets ranging in size from 16 to 8192 hours.<n>Specifically, we investigate how much additional training data is needed to achieve a target performance gain.
  arXiv  Detail & Related papers  (2025-04-06T03:23:48Z)
• Meta-Statistical Learning: Supervised Learning of Statistical Inference [59.463430294611626]
  This work demonstrates that the tools and principles driving the success of large language models (LLMs) can be repurposed to tackle distribution-level tasks.<n>We propose meta-statistical learning, a framework inspired by multi-instance learning that reformulates statistical inference tasks as supervised learning problems.
  arXiv  Detail & Related papers  (2025-02-17T18:04:39Z)
• OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesis [55.390060529534644]
  We propose OS-Genesis, a novel data synthesis pipeline for Graphical User Interface (GUI) agents.<n>Instead of relying on pre-defined tasks, OS-Genesis enables agents first to perceive environments and perform step-wise interactions.<n>We demonstrate that training GUI agents with OS-Genesis significantly improves their performance on highly challenging online benchmarks.
  arXiv  Detail & Related papers  (2024-12-27T16:21:58Z)
• End-to-End Meta-Bayesian Optimisation with Transformer Neural Processes [52.818579746354665]
  This paper proposes the first end-to-end differentiable meta-BO framework that generalises neural processes to learn acquisition functions via transformer architectures. We enable this end-to-end framework with reinforcement learning (RL) to tackle the lack of labelled acquisition data.
  arXiv  Detail & Related papers  (2023-05-25T10:58:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.