Planning-aligned Token Compression for Long-Context Autonomous Driving
Abstract Overview
This paper introduces COMPACT-VA, a planning-aligned token compression framework for long-context autonomous driving built on a conditional VQ-VAE and a hierarchical Q-former memory buffer. Instead of compressing visual history with rule-based temporal decay alone, the method conditions compression on both historical trajectory information and a learned driving-intent latent inferred from future trajectories during training and predicted from compressed observations at inference. The compressed memory and predicted latent are fed into a unified vision-action policy and optimized end to end so that retained tokens remain useful for downstream planning. The evaluation emphasizes memory-dependent driving scenarios such as four-way stops, stop/yield situations with occlusion, and unprotected turns, using behavior-oriented metrics rather than only trajectory displacement.
Novelty
The main novelty is aligning token compression directly with the planning objective, rather than treating compression as a separate heuristic efficiency module. The method combines hierarchical temporal compression with a conditional VQ-VAE latent representing driving intent, so the system learns which historical information is decision-critical for behavior.
Results
On stop-controlled intersection scenarios under comparable token budgets, COMPACT-VA reaches 68.3% go success rate, improving over the standard baseline (63.8%) and over compression without planning alignment (65.6%), while also reducing roll-through rate to about 7.0%. In closed-loop evaluation, it maintains broadly comparable general-driving performance while achieving 3.3x speedup and 2.7x memory reduction relative to uncompressed long-context processing.
Key Points
- COMPACT-VA compresses long visual context with a hierarchical Q-former memory that is conditioned on ego-history and a learned planning-intent latent.
- The paper targets memory-critical autonomous driving scenarios and evaluates them with behavior-focused metrics such as go success, stop success, roll-through rate, and stop position/duration errors.
- Experiments and ablations indicate that planning-aligned compression outperforms both uncompressed long-context input and compression without planning alignment, while also improving runtime and memory efficiency.