Jet-Long: Efficient Long-Context Extension with Dynamic Bifocal RoPE

• URL: http://arxiv.org/abs/2607.07740v2
• Date: Fri, 10 Jul 2026 03:53:58 GMT
• Title: Jet-Long: Efficient Long-Context Extension with Dynamic Bifocal RoPE
• Abstract summary: Jet-Long is a tuning-free zero-shot method that pairs a local RoPE-faithful window with a long-range window.<n>On Qwen3-1.7B/4B/8B up to 128K context, Jet-Long leads RULER by $+4.79$/$+2.18$/$+2.03$ pp over the strongest baseline at 1.7B/4B/8B.
• Score: 26.27800618469414
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern LLMs are increasingly deployed in long-context applications such as retrieval-augmented generation, repository-level coding, and agentic workflows whose accumulated reasoning and tool traces routinely push the input an order of magnitude past the pretraining window, making zero-shot context extension the dominant deployment path for open-weight checkpoints. The dominant zero-shot methods (YaRN, Self-Extend, DCA) fix a single rescaling factor up front, so an aggressive factor sacrifices short-context fidelity while a conservative one breaks down at long contexts; recent length-aware variants adapt the mapping, but with a fitted or distance-dependent schedule. We propose Jet-Long, a tuning-free zero-shot method that pairs a local RoPE-faithful window with a long-range window whose rescaling factor adapts dynamically to the current sequence length via a parameter-free analytic schedule, recovering the base model exactly at short inputs while extrapolating cleanly at long ones. An inclusion-exclusion attention merge and an on-the-fly RoPE correction rotation make the bifocal construction essentially free at inference; fused into a single CuTe kernel, long-context prefill reaches up to $1.39\times$ FA2 throughput on H100 (approaching the Hopper-only FA4), and single-batch generation incurs $\le 4\%$ overhead at every length. On Qwen3-1.7B/4B/8B up to 128K context, Jet-Long leads RULER by $+4.79$/$+2.18$/$+2.03$ pp over the strongest baseline at 1.7B/4B/8B, achieves the best overall accuracy on HELMET-RAG (a benchmark identified by HELMET as the most efficient predictor of downstream long-context performance) and attains the lowest PG-19 perplexity. Jet-Long also generalizes to hybrid attention architectures such as Jet-Nemotron for further long-context improvement without retraining, and remains hyperparameter-resilient for ease of deployment.

Related papers

• ECHO: Early-layer Collaborative Hierarchical Orchestration with Bonus Logits in Speculative Decoding [60.34475274870224]
  ECHO is a hierarchical dual-loop framework that exploits the functional asymmetry between LLM layers.<n>Within the inner loop, early-layer bonus logits drive rapid, multi-step draft-tree exploration at a minimal cost.<n>ECHO significantly boosts mean accepted tokens and achieves a 2.4$times$ to 2.9$times$ speedup.
  arXiv  Detail & Related papers  (2026-09-15T14:21:33Z)
• SpecRoll: Fast-Slow Verifier-Feedback Adaptation for Speculative Reinforcement Learning Rollouts [3.729816445527413]
  We introduce SpecRoll, a speculative rollout engine that preserves the target model's sampling distribution while adapting at two timescales.<n> Lightweight future-token heads generate parallel proposals, while a Reflex module uses delayed verifier feedback to performBounded, trajectory-local hidden-state corrections.<n>Across five models ranging from 1.5B to 14B, SpecRoll 1.26-2.15x generation speedup and 1.21-2.04x end-to-end speedup over vanilla GRPO.
  arXiv  Detail & Related papers  (2026-08-05T15:32:27Z)
• LongAttnComp: Cross-Family Context Compression for Long-Context Reasoning [2.1218260966433955]
  LongAttnComp is a long-context adaptation of AttnComp that fine-tunes a lightweight cross-attention scoring layer.<n>On InfiniteBench Code- Debug, LongAttnComp matches or exceeds full-context accuracy.<n>On LongBench v2, the two-stage recipe largely closes the Stage 1 gap on multi-document reasoning.
  arXiv  Detail & Related papers  (2026-05-31T16:40:36Z)
• EndPrompt: Efficient Long-Context Extension via Terminal Anchoring [62.81677226065374]
  We propose EndPrompt, a method that achieves effective context extension using only short training sequences.<n>We preserve the original short context as an intact first segment and append a brief terminal prompt as a second segment, assigning it positional indices near the target context length.<n>EndPrompt achieves an average RULER score of 76.03 and the highest average on LongBench, surpassing LCEG (72.24), LongLoRA (72.95), and full-length fine-tuning.
  arXiv  Detail & Related papers  (2026-05-14T09:00:03Z)
• Shuffle the Context: RoPE-Perturbed Self-Distillation for Long-Context Adaptation [86.62036852878354]
  Large language models (LLMs) increasingly operate in settings that require reliable long-context understanding.<n>We propose RoPE-Perturbed Self-Distillation, a training regularizer that improves positional robustness.<n> Experiments on long-context adaptation of Llama-3-8B and Qwen-3-4B demonstrate consistent gains on long-context benchmarks.
  arXiv  Detail & Related papers  (2026-04-15T18:46:35Z)
• VSPrefill: Vertical-Slash Sparse Attention with Lightweight Indexing for Long-Context Prefilling [0.0]
  Existing sparse attention methods face a trade-off among context adaptivity, sampling overhead, and fine-tuning costs.<n>We propose VSPrefill, a mechanism requiring lightweight training that uses the vertical-slash structural pattern in attention distributions.<n>VSPrefill preserves 98.35% of the full attention accuracy while delivering a 4.95x average speedup at a context length of 128k.
  arXiv  Detail & Related papers  (2026-03-03T09:24:58Z)
• Fast Forward: Accelerating LLM Prefill with Predictive FFN Sparsity [5.223181756178433]
  We introduce FastForward, a predictive sparsity framework that accelerates large language model (LLM) prefill through block-wise, context-aware FFN sparsity.<n>FastForward delivers up to 1.45$times$ compute-bound speedup at 50% FFN sparsity with $$ 6% accuracy loss compared to the dense baseline on LongBench.
  arXiv  Detail & Related papers  (2026-01-30T23:23:40Z)
• Length-Adaptive Interest Network for Balancing Long and Short Sequence Modeling in CTR Prediction [50.094751096858204]
  LAIN is a plug-and-play framework that incorporates sequence length as a conditioning signal to balance long- and short-sequence modeling.<n>Our work offers a general, efficient, and deployable solution to mitigate length-induced bias in sequential recommendation.
  arXiv  Detail & Related papers  (2026-01-27T03:14:20Z)
• Q-ROAR: Outlier-Aware Rescaling for RoPE Position Interpolation in Quantized Long-Context LLMs [0.9510848451801044]
  We show that combining PI with PTQ degrades accuracy due to coupled effects long context aliasing, dynamic range dilation, axis grid anisotropy, and shifting that induce position-dependent logit noise.<n>We propose Q-ROAR, a RoPE-aware, weight-only stabilization that groups RoPE dimensions into a few frequency bands and performs a small search over per-band scales for W_Q,W_K, with an optional symmetric variant to preserve logit scale.
  arXiv  Detail & Related papers  (2025-09-17T19:50:16Z)
• LongRoPE2: Near-Lossless LLM Context Window Scaling [46.936900701411965]
  LongRoPE2 is a novel approach that extends the effective context window of pre-trained large language models (LLMs) to the target length.<n>This is achieved by three contributions: (1) a hypothesis that insufficient training in higher RoPE dimensions contributes to the persistent out-of-distribution issues observed in existing methods; (2) an effective RoPE rescaling algorithm that adopts evolutionary search guided by "needle-driven" perplexity to address the insufficient training problem; and (3) a mixed context window training approach that fine-tunes model weights to adopt rescaled RoPE for long-context sequences.
  arXiv  Detail & Related papers  (2025-02-27T13:41:07Z)
• LongEmbed: Extending Embedding Models for Long Context Retrieval [87.60404151086715]
  This paper explores context window extension of embedding models, pushing the limit to 32k without requiring additional training. First, we examine the performance of current embedding models for long context retrieval on our newly constructed LongEmbed benchmark. Experiments show that training-free context window extension strategies like positionRo can effectively extend the context window of existing embedding models by several folds.
  arXiv  Detail & Related papers  (2024-04-18T11:29:23Z)
• HiRE: High Recall Approximate Top-$k$ Estimation for Efficient LLM Inference [68.59839755875252]
  HiRE comprises of two novel components: (i) a compression scheme to cheaply predict top-$k$ rows/columns with high recall, followed by full computation restricted to the predicted subset, and (ii) DA-TOP-$k$: an efficient multi-device approximate top-$k$ operator. We demonstrate that on a one billion parameter model, HiRE applied to both the softmax as well as feedforward layers, achieves almost matching pretraining and downstream accuracy, and speeds up inference latency by $1.47times$ on a single TPUv5e device.
  arXiv  Detail & Related papers  (2024-02-14T18:04:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.