REFACTOR-VLA: Unsupervised Library Learning of Typed Motor Programs

• URL: http://arxiv.org/abs/2609.01215v1
• Date: Tue, 01 Sep 2026 13:19:05 GMT
• Title: REFACTOR-VLA: Unsupervised Library Learning of Typed Motor Programs
• Abstract summary: Most vision-language-action (VLA) models are monolithic.<n>We introduce REFACTOR-VLA, a wake/sleepmodel system for learning reusable skills.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Most vision-language-action (VLA) models -- OpenVLA, $π_0$, RT-2, RDT-1B -- are monolithic: they emit raw motor commands or short action chunks without organizing behavior into reusable abstractions, so they degrade on long-horizon tasks and resist interpretation. Existing skill-discovery methods sidestep the core question of when two action sequences are behaviorally equivalent, either clustering contrastive embeddings or delegating the judgment to a language model uncalibrated to the robot's dynamics. We introduce REFACTOR-VLA, a wake/sleep system for learning reusable skills. Its sleep phase clusters motor-program fragments under a Behavioral-Equivalence Kernel (BEK) computed from rollouts of a learned latent world model $M_φ$; its wake phase emits typed lambda terms over a Hindley--Milner-inspired vocabulary, consumed by a library-conditioned rectified-flow action decoder. Abstractions are admitted only if they pass Minimum Description Length and return-preservation gates. On LIBERO we report two findings. First, enlarging the world model from 188M to 430M parameters worsened performance on 4 of 4 suites, so capacity alone does not help. Second, the training objective matters far more: adding an auxiliary supervised contrastive (InfoNCE) loss during world-model warmup substantially improves sleep-phase clustering, giving Normalized Mutual Information at $n=3$ seeds of $0.462 \pm 0.021$ (object), $0.867 \pm 0.025$ (spatial), $0.915 \pm 0.013$ (goal) and $0.754 \pm 0.010$ (LIBERO-10), and beating the strongest published baseline on all 4 suites by a mean $Δ= +0.184$. Across providers ($n=12$) the 95% bootstrap confidence interval for mean pairwise NMI is $[0.683, 0.729]$ (mean $0.705$). The sleep phase also yields the first real-LIBERO task-language library: the decoder uses 2 of 3 admitted abstractions and rewrites all 256 sampled demonstrations.

Related papers

• IMLE-VLA: Fast Single-Step Action Generation for Vision-Language-Action Policies [21.67040873250993]
  Vision-language-action (VLA) policies leverage pretrained vision-language backbones to achieve strong cross-task generalization.<n>We introduce IMLE-VLA, which replaces the iterative action head with a single-step conditional generator trained via conditional Implicit Maximum Likelihood Estimation (cIMLE)<n>When IMLE-VLA is applied to $_0.5$, it increases inference frequency 3.67x (55 Hz vs. 15 Hz) enabling up to 11x higher action throughput.
  arXiv  Detail & Related papers  (2026-09-10T00:00:32Z)
• EquiVLA: A General Framework for Rotationally Equivariant Vision-Language-Action Models [7.807191061951358]
  We present textscEquiVLA, the first general framework for end-to-end $mathrmSO(2)$-equivariant VLA models.<n>textscEquiVLA is applicable to any architecture coupling a frozen vision-language backbone with a flow-matching Diffusion Transformer action head.
  arXiv  Detail & Related papers  (2026-06-18T04:36:57Z)
• Toward Compiler World Models: Learning Latent Dynamics for Efficient Tensor Program Search [9.85275171877854]
  We propose an evaluator that schedules evaluation as action-conditioned latent dynamics over program states.<n>Our method improves representative-subgraph latency over Ansor by 1.37$times$ on GPU and 1.54$times$ on CPU under the same 64-trial budget.<n>It also accelerates full-model inference over PyTorch/PyTorch-opt(cuDNN) by 4.61$times$/3.67$times$ geometric mean.
  arXiv  Detail & Related papers  (2026-06-08T10:17:27Z)
• Which Defense Closes Which Threat? Attributing OWASP-LLM-Top-10 Coverage and Its Brittleness Under Paraphrasing [51.56484100374058]
  Production LLM applications stack several defense families -- refusal-phrase filters, tokenbudget controls, model allowlists, rate limits, tool-registry authentication -- yetBAS benchmarks report a single aggregate coverage number.<n>We add four-LLM-Top-10-aware agents to a 21-agent baseline scanner and target a lattice of four synthetic LLM endpoints.
  arXiv  Detail & Related papers  (2026-06-01T19:39:25Z)
• What Do EEG Foundation Models Capture from Human Brain Signals? [64.48249643001402]
  Modern EEG foundation models learn directly from raw signals via self-supervised pretraining.<n>We decompose into three sub-questions: emphwhat does the model learn, emphwhat does the model use, and emphhow much can be explained<n>The audit covers three foundation models (CSBrain, CBraMod, LaBraM), five clinical tasks (MDD, Stress, ISRUC-Sleep, TUSL, Siena), and a 6-family 63-feature lexicon.
  arXiv  Detail & Related papers  (2026-05-12T01:57:53Z)
• Introspection Fine-Tuning (IFT): Training Small LLMs to Introspect [0.0]
  We investigate whether small language models can detect and report on perturbations their own internal activations.<n>We first show that the binary detection paradigm used in prior work is confounded in small models, as steering biases the model toward affirmative responses regardless of the question content.<n>We then introduce emphIntrospection Fine-Tuning (IFT), supervised fine-tuning on sentence-localization examples constructed from the model's own perturbed forward passes.
  arXiv  Detail & Related papers  (2026-05-08T05:30:26Z)
• Understanding Asynchronous Inference Methods for Vision-Language-Action Models [0.0]
  Vision-Language-Action (VLA) models offer a promising path to generalist robot control, but their inference latency causes observation staleness when generated actions are executed asynchronously.<n>Several methods have been proposed concurrently to mitigate this problem: inference-time inpainting (IT-RTC), training-time delay simulation (TT-RTC), future-state-aware conditioning (VLASH), and lightweight residual correction (A2C2)<n>We present a systematic comparison of these four methods under controlled conditions.
  arXiv  Detail & Related papers  (2026-05-04T18:01:15Z)
• Large Language Models as Optimization Controllers: Adaptive Continuation for SIMP Topology Optimization [4.365822392824561]
  We present a framework in which a large language model acts as an online controller for SIMP optimization.<n>A hard grayness gate prevents premature binarization, and a meta-optimization loop uses a second pass to tune the agent's call frequency.<n>The agent achieves the final compliance on every benchmark: $-5.7%$ to $-1% relative to the fixed baseline, with all solutions fully binary.
  arXiv  Detail & Related papers  (2026-03-26T07:14:31Z)
• INC: An Indirect Neural Corrector for Auto-Regressive Hybrid PDE Solvers [61.84396402100827]
  We propose the Indirect Neural Corrector ($mathrmINC$), which integrates learned corrections into the governing equations.<n>$mathrmINC$ reduces the error amplification on the order of $t-1 + L$, where $t$ is the timestep and $L$ the Lipschitz constant.<n>We test $mathrmINC$ in extensive benchmarks, covering numerous differentiable solvers, neural backbones, and test cases ranging from a 1D chaotic system to 3D turbulence.
  arXiv  Detail & Related papers  (2025-11-16T20:14:28Z)
• Runaway is Ashamed, But Helpful: On the Early-Exit Behavior of Large Language Model-based Agents in Embodied Environments [54.67512489842682]
  Large language models (LLMs) have demonstrated strong planning and decision-making capabilities in complex embodied environments.<n>We take a first step toward exploring the early-exit behavior for LLM-based agents.
  arXiv  Detail & Related papers  (2025-05-23T08:23:36Z)
• FLARE: Robot Learning with Implicit World Modeling [87.81846091038676]
  $textbfFLARE$ integrates predictive latent world modeling into robot policy learning.<n>$textbfFLARE$ achieves state-of-the-art performance, outperforming prior policy learning baselines by up to 26%.<n>Our results establish $textbfFLARE$ as a general and scalable approach for combining implicit world modeling with high-frequency robotic control.
  arXiv  Detail & Related papers  (2025-05-21T15:33:27Z)
• From Continual Learning to SGD and Back: Better Rates for Continual Linear Models [50.11453013647086]
  We analyze the forgetting, i.e., loss on previously seen tasks, after $k$ iterations.<n>We develop novel last-iterate upper bounds in the realizable least squares setup.<n>We prove for the first time that randomization alone, with no task repetition, can prevent catastrophic in sufficiently long task sequences.
  arXiv  Detail & Related papers  (2025-04-06T18:39:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.