Cross-Lingual Token Arbitrage: Optimizing Code Agent Context Windows via Local LLM Preprocessing

• URL: http://arxiv.org/abs/2606.03618v1
• Date: Tue, 02 Jun 2026 13:17:45 GMT
• Title: Cross-Lingual Token Arbitrage: Optimizing Code Agent Context Windows via Local LLM Preprocessing
• Abstract summary: Two pathologies of raw human input drive much of this overhead: tokenization inefficiency for non-English text and structural entropy in conversational prompts.<n>Existing approaches act reactively by compressing already-bloated contexts or intervening after failures occur.<n>We introduce a pre-flight, edge-side prompt-rewriting that operates between the developer and the cloud agent.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: AI-assisted coding agents are bottlenecked by input-token cost. Two pathologies of raw human input drive much of this overhead: tokenization inefficiency for non-English text and structural entropy in conversational prompts. Existing approaches act reactively by compressing already-bloated contexts or intervening after failures occur. We introduce a pre-flight, edge-side prompt-rewriting middleware that operates between the developer and the cloud agent. A local Llama 3.2 (3B) model performs cross-lingual translation into English, structural rewriting into a compact task-oriented format, and regex-validated rewrite-with-fallback safeguards to ensure the optimized prompt is never larger than the original. We evaluate on OMH-Polyglot, a multilingual coding benchmark spanning Turkish, Arabic, Chinese, and code-switched specifications. Across three commercial LLM backends, the middleware reduces prompt tokens by 34-47 percent and total tokens by up to 18.8 percent while preserving or improving task accuracy. Ablation studies show that gains arise primarily from the rewriting stage rather than simple function-name extraction. Compared with LLMLingua-2 at matched compression rates, our method consistently achieves superior OckScore performance across all evaluated backends. These results demonstrate that proactive prompt optimization can substantially reduce inference costs without sacrificing coding quality.

Related papers

• SEDCoT: Enhancing LLM-Based COBOL Code Translation via Symbolic Execution and Delta Debugging [49.898541267264115]
  Maintenance is increasingly challenging due to outdated technologies, sparse documentation, and developer retirement.<n>Traditional rule-based transcompilers yield outputs that are difficult to read and maintain.<n>We propose SEDCoT, a novel-to-C translation framework.
  arXiv  Detail & Related papers  (2026-07-05T03:07:10Z)
• CANDLE: Character-level Arabic Noise Deduplication using Lightweight Encoder [1.0399316033225514]
  We present CANDLE, a lightweight system for character-level Arabic noise deduplication.<n>At the heart of CANDLE is a novel application of Connectionist Temporal Classification (CTC) to this task.<n>The CTC model achieves a Sentence Error Rate (SER) as low as $5.37%$ and consistently outperforms a classification-based baseline by a large margin.
  arXiv  Detail & Related papers  (2026-06-23T16:17:24Z)
• SimSD: Simple Speculative Decoding in Diffusion Language Models [61.33773959352141]
  Diffusion large language models (dLLMs) offer faster inference through parallel or blockwise decoding.<n>Their masked language modeling formulation remains incompatible with standard token-level speculative decoding.<n>We propose a speculative decoding algorithm for dLLMs, named SimSD, that equips dLLMs with temporally valid token-level contexts.<n>Our method achieves up to 7.46x higher decoding throughput while maintaining and even improving average generation quality.
  arXiv  Detail & Related papers  (2026-06-01T17:46:46Z)
• Hierarchical Policy Optimization for Simultaneous Translation of Unbounded Speech [50.45710815530982]
  Simultaneous speech translation (SST) generates translations while receiving partial speech input.<n>Recent advances show that large language models (LLMs) can substantially improve SST quality, but at the cost of high computational overhead.<n>We propose a Hierarchical Policy Optimization (HPO) approach that trains post-train models on imperfect SFT data.<n> Experiments on English to Chinese/German/Japanese demonstrate improvements of over +7 COMET score and +1.25 MetricX score at a latency of 1.5 seconds.
  arXiv  Detail & Related papers  (2026-04-22T19:43:51Z)
• Languages are Modalities: Cross-Lingual Alignment via Encoder Injection [0.8461674097042394]
  We present a compute efficient language-as-modality method that conditions an instruction-tuned decoder without changing the tokenizer or retraining the decoder.<n>LLINK substantially improves bilingual retrieval and achieves 81.3% preference over the base model.<n>We find that improvements can be attributed to reduced tokenization inflation and a stronger cross lingual alignment.
  arXiv  Detail & Related papers  (2025-10-31T07:43:21Z)
• Finish First, Perfect Later: Test-Time Token-Level Cross-Validation for Diffusion Large Language Models [47.5976588836299]
  Diffusion large language models (dLLMs) offer advantages such as accelerated parallel decoding and bidirectional context modeling.<n>The vanilla decoding strategy in discrete dLLMs suffers from a critical limitation: once a token is accepted, it can no longer be revised in subsequent steps.<n>We propose Tolerator, a training-free decoding strategy that leverages cross-validation among predicted tokens.
  arXiv  Detail & Related papers  (2025-10-06T17:56:46Z)
• LLM Encoder vs. Decoder: Robust Detection of Chinese AI-Generated Text with LoRA [4.104443734934105]
  We compare encoder-based Transformers (Chinese BERT-large and RoBERTa-wwm-ext-large), a decoder-only LLM (Alibaba's Qwen2.5-7B/Deep-R1-Distill-Qwen-7B fine-tuned via Low-Rank Adaptation, LoRA), and a FastText baseline.<n>Experiments reveal that although encoder models nearly memorize training data, they suffer significant performance degradation under distribution shifts.
  arXiv  Detail & Related papers  (2025-08-31T07:51:22Z)
• Fast Controlled Generation from Language Models with Adaptive Weighted Rejection Sampling [90.86991492288487]
  evaluating constraint on every token can be prohibitively expensive.<n> LCD can distort the global distribution over strings, sampling tokens based only on local information.<n>We show that our approach is superior to state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-04-07T18:30:18Z)
• Multilingual Contrastive Decoding via Language-Agnostic Layers Skipping [60.458273797431836]
  Decoding by contrasting layers (DoLa) is designed to improve the generation quality of large language models. We find that this approach does not work well on non-English tasks. Inspired by previous interpretability work on language transition during the model's forward pass, we propose an improved contrastive decoding algorithm.
  arXiv  Detail & Related papers  (2024-07-15T15:14:01Z)
• LeTI: Learning to Generate from Textual Interactions [60.425769582343506]
  We explore LMs' potential to learn from textual interactions (LETI) that not only check their correctness with binary labels but also pinpoint and explain errors in their outputs through textual feedback. Our focus is the code generation task, where the model produces code based on natural language instructions. LETI iteratively fine-tunes the model, using the objective LM, on a concatenation of natural language instructions, LM-generated programs, and textual feedback.
  arXiv  Detail & Related papers  (2023-05-17T15:53:31Z)
• RetroMAE v2: Duplex Masked Auto-Encoder For Pre-Training Retrieval-Oriented Language Models [3.4523793651427113]
  We propose duplex masked auto-encoder, a.k.a. DupMAE, which targets on improving the semantic representation capacity for contextualized embeddings of both [] and ordinary tokens. DupMAE is simple but empirically competitive: with a small decoding cost, it substantially contributes to the model's representation capability and transferability.
  arXiv  Detail & Related papers  (2022-11-16T08:57:55Z)
• Incorporating a Local Translation Mechanism into Non-autoregressive Translation [28.678752678905244]
  We introduce a novel local autoregressive translation mechanism into non-autoregressive translation (NAT) models. For each target decoding position, instead of only one token, we predict a short sequence of tokens in an autoregressive way. We design an efficient merging algorithm to align and merge the out-put pieces into one final output sequence.
  arXiv  Detail & Related papers  (2020-11-12T00:32:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.