FIRE-VLA: Failure-Informed Self-Evolution for Vision-Language-Action Models in Autonomous Driving

• URL: http://arxiv.org/abs/2608.13395v1
• Date: Thu, 13 Aug 2026 15:53:24 GMT
• Title: FIRE-VLA: Failure-Informed Self-Evolution for Vision-Language-Action Models in Autonomous Driving
• Abstract summary: Group relative policy optimization learns from reward differences within each rollout group.<n>Fire-VLA converts unresolved failures into privileged supervision for the next policy.<n>On 6,019 examples from 150 held-out nuScenes scenes, FIRE-VLA retains comparable single-sample planning.
• Score: 1.4298580363875282
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reinforcement learning improves autonomous-driving vision-language-action (VLA) models by evaluating trajectories sampled from the current policy. Group relative policy optimization (GRPO) learns from reward differences within each rollout group. When all sampled trajectories are poor, this relative signal can rank failures without identifying behavior outside the failed region. We introduce FIRE-VLA, a failure-informed self-evolution framework that converts such unresolved failures into privileged supervision for the next policy. Low-reward, low-diversity groups trigger self-distillation from a frozen round-start copy of the same model. Teacher and student have the same parameter scale, but only the teacher observes the hidden future trajectory. Supervision follows the student's generated prefix and is restricted to answer tokens, while GRPO remains active for every group. The updated policy supplies the teacher for the next round, allowing the routed failure distribution to change with the policy without requiring a larger external teacher. Starting from the same Qwen2.5-VL-3B SFT checkpoint, the comparison matches student rollout and policy-update counts. On 6,019 examples from 150 held-out nuScenes scenes, FIRE-VLA retains comparable single-sample planning, reduces G=4 mean L2 from 1.848 to 1.500 m, and lowers evaluation-persistent failure prevalence from 13.03% to 11.20%. The reduction in mean error arises mainly from rare severe rollouts rather than uniform improvement across ordinary trajectories.

Related papers

• ReDraft, Don't Just Distill: Reference-Driven Revision for Continual VLLM Post-Training [66.72708893922605]
  Post-training of large multimodal models should add new capabilities while preserving those from pre-training.<n>SFT gives explicit target supervision that learns a task from near-zero accuracy, but its off-policy targets move the model far enough to cause forgetting.<n>We introduce ReDraft, which obtains both from the model's own failures.
  arXiv  Detail & Related papers  (2026-09-15T04:59:03Z)
• Prism-GRPO: Faster VLA Policy Optimization via Splitting Same-outcome Groups [14.712224182088327]
  Under binary success rewards, groups whose rollouts all succeed or all fail have zero advantage and are discarded by dynamic sampling.<n>We introduce Prism-GRPO, which augments binary outcome reward with a weighted trajectory-level execution-quality score.<n>Quality scores can be derived from simulator contacts, executed actions, or visual observations, avoiding task-specific progress rewards.
  arXiv  Detail & Related papers  (2026-08-18T06:44:16Z)
• I-SDPO: Instance-Level Adaptive Self-Distillation Policy Optimization [5.536267204193318]
  Group Relative Policy Optimization learns from reward differences within a rollout group, but receives no useful relative signal when every sampled response is incorrect.<n> Privileged self-distillation can fill this gap with dense token supervision, yet applying it throughout training creates a different failure mode.<n>We introduce I-SDPO (Instance-Level Adaptive Self-Distillation Policy Optimization), which treats teacher reliance as capability-dependent.
  arXiv  Detail & Related papers  (2026-08-13T08:37:24Z)
• WDL-OPD: Weak-Driven On-Policy Distillation via Mixture-Constrained Co-Training [48.290951447286744]
  WDL-OPD is a mixture-constrained co-training method with two trainable policies.<n>An anchor policy generates every rollout, an auxiliary policy evaluates the same visited states, and a geometric mixture of their token distributions is matched to a frozen teacher by reverse KL.<n>We show that freezing the auxiliary recovers an anchor-plus-contrast proxy target closely related to OPD$2$ and W2S-OPD, whereas joint training creates branch-level degrees of freedom that a static delta cannot express.
  arXiv  Detail & Related papers  (2026-08-10T11:22:53Z)
• Learning More from Less: Reinforcement Learning from Hindsight [19.802783289526598]
  Reinforcement learning (RL) is increasingly used to post-train vision-language-action (VLA) models.<n>We present Learning from Hindsight (LfH), which brings hindsight relabeling to RL post-training ofVLAs by scoring failed rollouts against the tasks they actually achieved.
  arXiv  Detail & Related papers  (2026-07-10T02:17:41Z)
• RL Forgets! Towards Continual Policy Optimization [57.4336338996653]
  Continual post-training is becoming a central paradigm for adapting vision-language models to evolving tasks.<n>Recent work has increasingly favored reinforcement learning over supervised fine-tuning, driven by the belief that reinforcement learning is inherently less prone to forgetting.<n>We propose Continual Policy Optimization (CPO), a replay-free framework grounded in a prior-task behavioral KL objective.
  arXiv  Detail & Related papers  (2026-07-05T15:46:11Z)
• Deterministic Decisions for High-Stakes AI. A Zero-Egress Pipeline with the Deployability of RAG and the Accuracy of Machine Learning [0.0]
  Intervention bias is a previously unquantified failure mode of large-language-model educational advisory agents.<n>We show that supervised policy learning eliminates this bias.<n>We also show an Evaluation Gap: LLM-as-judge scoring is blind to intervention bias, rewarding fluent over-prescription rather than decision quality.
  arXiv  Detail & Related papers  (2026-06-28T08:58:17Z)
• CIGPO: Contextual Information-Gain Policy Optimization for Multi-Turn Evidence-Reading LLM Agents [1.4298580363875282]
  Training multi-turn evidence-reading agents with outcome-only reinforcement learning is unstable because intermediate turns receive little direct credit.<n>In HotpotQA experiments with Qwen2.5-3B-Instruct, GRPO initially improves (standard F1 0.430) but subsequently collapses to 100% format-violating outputs.<n>We propose a variance-textual strategy: by assigning per-turn rewards to intermediate evidence-reading turns, we prevent the group reward distribution from collapsing to a single value.
  arXiv  Detail & Related papers  (2026-06-26T11:50:29Z)
• Self-Distilled Policy Gradient [51.95967256628261]
  On-policy self-distillation is a promising source of dense supervision for sparse-reward reinforcement learning.<n>We propose SDPG, a self-distilled policy-gradient framework that combines group-relative verifier advantages with normalized standard deviation.
  arXiv  Detail & Related papers  (2026-06-02T02:31:13Z)
• Teacher-Guided Policy Optimization for LLM Distillation [90.49982387646861]
  Teacher-Guided Policy Optimization (TGPO) is an on-policy algorithm that incorporates dense directional guidance by leveraging teacher predictions conditioned on the student's rollout.<n>Experiments on complex reasoning benchmarks demonstrate that TGPO significantly outperforms standard baselines and is robust to different teachers.
  arXiv  Detail & Related papers  (2026-05-13T09:20:03Z)
• Improving Search Agent with One Line of Code [68.58667107354253]
  Tool-based Agentic Reinforcement Learning (TARL) has emerged as a promising paradigm for training search agents to interact with external tools.<n>We propose textbfSearch textbfAgent textbfPolicy textbfOptimization (textbfSAPO), which stabilizes training via a conditional token-level KL constraint.
  arXiv  Detail & Related papers  (2026-03-10T04:07:39Z)
• CARE What Fails: Contrastive Anchored-REflection for Verifiable Multimodal [84.71254539482369]
  Group-relative reinforcement learning with verifiable rewards (RLVR) often wastes the most informative data it already has the failures.<n>We present CARE, a failure-centric post-training framework for multimodal reasoning that turns errors into supervision.<n> CARE improves accuracy and training smoothness while explicitly increasing the share of learning signal that comes from failures.
  arXiv  Detail & Related papers  (2025-12-22T16:34:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.