Action Chunking and Exploratory Data Collection Yield Exponential Improvements in Behavior Cloning for Continuous Control

• URL: http://arxiv.org/abs/2507.09061v4
• Date: Mon, 03 Nov 2025 23:01:49 GMT
• Title: Action Chunking and Exploratory Data Collection Yield Exponential Improvements in Behavior Cloning for Continuous Control
• Abstract summary: Action-chunking (predicting sequences of actions in open-loop) and exploratory augmentation of expert demonstrations are studied.<n>Our results identify control-theoretic stability as the key mechanism underlying the benefits of these interventions.<n>On the theoretical side, we demonstrate that the control-theoretic lens provides fine-grained insights into how compounding error arises.
• Score: 24.901172157291096
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper presents a theoretical analysis of two of the most impactful interventions in modern learning from demonstration in robotics and continuous control: the practice of action-chunking (predicting sequences of actions in open-loop) and exploratory augmentation of expert demonstrations. Though recent results show that learning from demonstration, also known as imitation learning (IL), can suffer errors that compound exponentially with task horizon in continuous settings, we demonstrate that action chunking and exploratory data collection circumvent exponential compounding errors in different regimes. Our results identify control-theoretic stability as the key mechanism underlying the benefits of these interventions. On the empirical side, we validate our predictions and the role of control-theoretic stability through experimentation on popular robot learning benchmarks. On the theoretical side, we demonstrate that the control-theoretic lens provides fine-grained insights into how compounding error arises, leading to tighter statistical guarantees on imitation learning error when these interventions are applied than previous techniques based on information-theoretic considerations alone.

Related papers

• Learning Interaction Kernels from Collective Steady States [4.349345557804093]
  We propose a learning procedure for system identification in interacting particle systems from single-snapshot observations of collective behaviors.<n>We test our learning procedure on a variety of representative models with steady-state and quasi-stationary patterns, where collective behaviors encode implicit information about the interaction mechanisms.
  arXiv  Detail & Related papers  (2026-09-10T01:53:57Z)
• Forgetting, plasticity, and co-observation: a third facet of continual learning [35.010469869321305]
  Efficient continual learning remains a fundamental challenge for deep neural networks.<n>We show that catastrophic forgetting and loss of plasticity are widely considered the primary obstacles to overcome.<n>We highlight data co-observation as a distinct factor influencing continual learning performance.
  arXiv  Detail & Related papers  (2026-08-19T10:59:59Z)
• Test-Time Learning of Causal Structure from Interventional Data [50.06913286558919]
  We propose TICL (Test-time Interventional Causal Learning), a novel method that synergizes Test-Time Training with Joint Causal Inference.<n>Specifically, we design a self-augmentation strategy to generate instance-specific training data at test time, effectively avoiding distribution shifts.<n>By integrating joint causal inference, we developed a PC-inspired two-phase supervised learning scheme, which effectively leverages self-augmented training data while ensuring theoretical identifiability.
  arXiv  Detail & Related papers  (2026-02-22T11:23:05Z)
• Belief Dynamics Reveal the Dual Nature of In-Context Learning and Activation Steering [22.666436755894328]
  Large language models (LLMs) can be controlled at inference time through prompts (in-context learning) and internal activations (activation steering)<n>This work offers a unified account of prompt-based and activation-based control of LLM behavior, and a methodology for empirically predicting the effects of these interventions.
  arXiv  Detail & Related papers  (2025-11-01T16:46:03Z)
• An Augmentation-Aware Theory for Self-Supervised Contrastive Learning [25.01234368914713]
  We propose an augmentation-aware error bound for self-supervised contrastive learning.<n>We show that the supervised risk is bounded not only by the unsupervised risk, but also explicitly by a trade-off induced by data augmentation.
  arXiv  Detail & Related papers  (2025-05-28T10:18:20Z)
• The Pitfalls of Imitation Learning when Actions are Continuous [33.44344966171865]
  We study the problem of imitating an expert demonstrator in a continuous state-and-action control system.<n>We show that, even if the dynamics satisfy a control-theoretic property called exponentially stability, any smooth, deterministic imitator policy necessarily suffers error.
  arXiv  Detail & Related papers  (2025-03-12T18:11:37Z)
• ACTIVA: Amortized Causal Effect Estimation via Transformer-based Variational Autoencoder [7.987204219322316]
  We propose ACTIVA, a conditional variational autoencoder architecture for amortized causal inference.<n>ACTIVA learns a latent representation conditioned on observational inputs and intervention queries, enabling zero-shot inference.<n>We provide theoretical insights showing that ACTIVA predicts interventional distributions as mixtures over observationally equivalent causal models.
  arXiv  Detail & Related papers  (2025-03-03T08:28:25Z)
• Logarithmic Regret for Nonlinear Control [5.473636587010879]
  We address the problem of learning to control an unknown nonlinear dynamical system through sequential interactions.<n>Motivated by high-stakes applications in which mistakes can be catastrophic, we study situations where it is possible for fast sequential learning to occur.
  arXiv  Detail & Related papers  (2025-01-17T15:42:42Z)
• Counterfactual Generative Modeling with Variational Causal Inference [1.9287470458589586]
  We present a novel variational Bayesian causal inference framework to handle counterfactual generative modeling tasks.<n>In experiments, we demonstrate the advantage of our framework compared to state-of-the-art models in counterfactual generative modeling.
  arXiv  Detail & Related papers  (2024-10-16T16:44:12Z)
• Temporal-Difference Variational Continual Learning [77.92320830700797]
  We propose new learning objectives that integrate the regularization effects of multiple previous posterior estimations.<n>Our approach effectively mitigates Catastrophic Forgetting, outperforming strong Variational CL methods.
  arXiv  Detail & Related papers  (2024-10-10T10:58:41Z)
• A Mathematical Model of the Hidden Feedback Loop Effect in Machine Learning Systems [44.99833362998488]
  We introduce a repeated learning process to jointly describe several phenomena attributed to unintended hidden feedback loops. A distinctive feature of such repeated learning setting is that the state of the environment becomes causally dependent on the learner itself over time. We present a novel dynamical systems model of the repeated learning process and prove the limiting set of probability distributions for positive and negative feedback loop modes.
  arXiv  Detail & Related papers  (2024-05-04T17:57:24Z)
• Can Active Sampling Reduce Causal Confusion in Offline Reinforcement Learning? [58.942118128503104]
  Causal confusion is a phenomenon where an agent learns a policy that reflects imperfect spurious correlations in the data. This phenomenon is particularly pronounced in domains such as robotics. In this paper, we study causal confusion in offline reinforcement learning.
  arXiv  Detail & Related papers  (2023-12-28T17:54:56Z)
• Interpretable Imitation Learning with Dynamic Causal Relations [65.18456572421702]
  We propose to expose captured knowledge in the form of a directed acyclic causal graph. We also design this causal discovery process to be state-dependent, enabling it to model the dynamics in latent causal graphs. The proposed framework is composed of three parts: a dynamic causal discovery module, a causality encoding module, and a prediction module, and is trained in an end-to-end manner.
  arXiv  Detail & Related papers  (2023-09-30T20:59:42Z)
• A Double Machine Learning Approach to Combining Experimental and Observational Data [58.05402364136958]
  We propose a double machine learning approach to combine experimental and observational studies.<n>Our framework proposes a falsification test for external validity and ignorability under milder assumptions.
  arXiv  Detail & Related papers  (2023-07-04T02:53:11Z)
• CausalBench: A Large-scale Benchmark for Network Inference from Single-cell Perturbation Data [61.088705993848606]
  We introduce CausalBench, a benchmark suite for evaluating causal inference methods on real-world interventional data. CaulBench incorporates biologically-motivated performance metrics, including new distribution-based interventional metrics.
  arXiv  Detail & Related papers  (2022-10-31T13:04:07Z)
• Imitating, Fast and Slow: Robust learning from demonstrations via decision-time planning [96.72185761508668]
  Planning at Test-time (IMPLANT) is a new meta-algorithm for imitation learning. We demonstrate that IMPLANT significantly outperforms benchmark imitation learning approaches on standard control environments.
  arXiv  Detail & Related papers  (2022-04-07T17:16:52Z)
• SurvITE: Learning Heterogeneous Treatment Effects from Time-to-Event Data [83.50281440043241]
  We study the problem of inferring heterogeneous treatment effects from time-to-event data. We propose a novel deep learning method for treatment-specific hazard estimation based on balancing representations.
  arXiv  Detail & Related papers  (2021-10-26T20:13:17Z)
• Shaking the foundations: delusions in sequence models for interaction and control [45.34593341136043]
  We show that sequence models "lack the understanding of the cause and effect of their actions" leading them to draw incorrect inferences due to auto-suggestive delusions. We show that in supervised learning, one can teach a system to condition or intervene on data by training with factual and counterfactual error signals respectively.
  arXiv  Detail & Related papers  (2021-10-20T23:31:05Z)
• Social NCE: Contrastive Learning of Socially-aware Motion Representations [87.82126838588279]
  Experimental results show that the proposed method dramatically reduces the collision rates of recent trajectory forecasting, behavioral cloning and reinforcement learning algorithms. Our method makes few assumptions about neural architecture designs, and hence can be used as a generic way to promote the robustness of neural motion models.
  arXiv  Detail & Related papers  (2020-12-21T22:25:06Z)
• Bridging the Imitation Gap by Adaptive Insubordination [88.35564081175642]
  We show that when the teaching agent makes decisions with access to privileged information, this information is marginalized during imitation learning. We propose 'Adaptive Insubordination' (ADVISOR) to address this gap. ADVISOR dynamically weights imitation and reward-based reinforcement learning losses during training, enabling on-the-fly switching between imitation and exploration.
  arXiv  Detail & Related papers  (2020-07-23T17:59:57Z)
• State-Only Imitation Learning for Dexterous Manipulation [63.03621861920732]
  In this paper, we explore state-only imitation learning. We train an inverse dynamics model and use it to predict actions for state-only demonstrations. Our method performs on par with state-action approaches and considerably outperforms RL alone.
  arXiv  Detail & Related papers  (2020-04-07T17:57:20Z)
• Estimating the Effects of Continuous-valued Interventions using Generative Adversarial Networks [103.14809802212535]
  We build on the generative adversarial networks (GANs) framework to address the problem of estimating the effect of continuous-valued interventions. Our model, SCIGAN, is flexible and capable of simultaneously estimating counterfactual outcomes for several different continuous interventions. To address the challenges presented by shifting to continuous interventions, we propose a novel architecture for our discriminator.
  arXiv  Detail & Related papers  (2020-02-27T18:46:21Z)
• Metric-Based Imitation Learning Between Two Dissimilar Anthropomorphic Robotic Arms [29.08134072341867]
  One major challenge in imitation learning is the correspondence problem. We introduce a distance measure between dissimilar embodiments. We find that the measure is well suited for describing the similarity between embodiments and for learning imitation policies by distance.
  arXiv  Detail & Related papers  (2020-02-25T19:47:19Z)
• Nonparametric inference for interventional effects with multiple mediators [0.0]
  We provide theory that allows for more flexible, possibly machine learning-based, estimation techniques. We demonstrate multiple robustness properties of the proposed estimators. Our work thus provides a means of leveraging modern statistical learning techniques in estimation of interventional mediation effects.
  arXiv  Detail & Related papers  (2020-01-16T19:05:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.