OneLatent: Single-Token Compression for Visual Latent Reasoning

• URL: http://arxiv.org/abs/2602.13738v1
• Date: Sat, 14 Feb 2026 12:03:28 GMT
• Title: OneLatent: Single-Token Compression for Visual Latent Reasoning
• Abstract summary: We present textbfOneLatent, a framework that compresses intermediate reasoning into a single latent token via supervision from rendered CoT images and DeepSeek-OCR hidden states.<n>Across benchmarks, OneLatent reduces average output length by $11times$ with only a $2.21%$ average accuracy drop relative to textual CoT.<n>On long-chain logical reasoning, OneLatent reaches $99.80%$ on ProntoQA and $97.80%$ on ProsQA with one latent token, with compression up to $87.4times$.
• Score: 12.825780556445892
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Chain-of-thought (CoT) prompting improves reasoning but often increases inference cost by one to two orders of magnitude. To address these challenges, we present \textbf{OneLatent}, a framework that compresses intermediate reasoning into a single latent token via supervision from rendered CoT images and DeepSeek-OCR hidden states. By rendering textual steps into images, we obtain a deterministic supervision signal that can be inspected and audited without requiring the model to output verbose textual rationales. Across benchmarks, OneLatent reduces average output length by $11\times$ with only a $2.21\%$ average accuracy drop relative to textual CoT, while improving output token contribution (OTC) by $6.8\times$. On long-chain logical reasoning, OneLatent reaches $99.80\%$ on ProntoQA and $97.80\%$ on ProsQA with one latent token, with compression up to $87.4\times$, supporting compression-constrained generalization.

Related papers

• STAC: Selective Spatiotemporal Aggregation and Compression for Video Reasoning Segmentation [60.597357847359255]
  Video segmentation demands pixel-accurate object tracking across hundreds of frames under complex natural language queries.<n>Existing methods address this through token compression, yet typically operate on features lacking temporal context.<n>State-space models resolve this constraint, as their linear recurrence selectively conditions each token on temporal context at $mathcalO(T)$ cost compression to reduce.
  arXiv  Detail & Related papers  (2026-07-03T03:28:25Z)
• ReasonAlloc: Hierarchical Decoding-Time KV Cache Budget Allocation for Reasoning Models [51.947287846779545]
  ReasonAlloc recasts decoding-time KV compression as a hierarchical budget allocation problem.<n>An offline layer-wise preallocation strategy captures an architecture-driven demand pattern.<n>An online head-wise strategy reallocates resources during decoding to information-rich heads based on real-time utility.
  arXiv  Detail & Related papers  (2026-06-09T17:44:23Z)
• Entropy Gate: Entropy Quenching for Near-Lossless Token Compression in LLM Pipelines [0.0]
  LLM pipelines waste substantial token budgets on low-information content.<n>We introduce Entropy Gate, a token compression framework applying entropy quenching.<n>The framework is stateless, model-agnostic, and deploys as an OpenAI-compatible HTTP proxy.
  arXiv  Detail & Related papers  (2026-06-02T14:55:02Z)
• Pair-In, Pair-Out: Latent Multi-Token Prediction for Efficient LLMs [19.38885798529711]
  We propose textbfPair-In, Pair-Out (PIPO), which unifies both sides by viewing a latent compressor and an MTP head as mirror-image operations.<n>PIPO trains a lightweight confidence head that decides whether draft tokens should be accepted.<n>Experiments on AIME 2025, GPQA-Diamond, LiveCodeBench v6, and LongBench v2 with Qwen3.5-4B and 9B backbones show that PIPO improves pass@4 over regular decoding by up to $+7.15$ points.
  arXiv  Detail & Related papers  (2026-05-26T16:31:45Z)
• LaTER: Efficient Test-Time Reasoning via Latent Exploration and Explicit Verification [35.08680804423239]
  Chain-of-thought (CoT) reasoning improves large language models (LLMs) on difficult tasks, but it also makes inference expensive.<n>We propose Latent-Then-Explicit Reasoning (LaTER), a two-stage paradigm that first performs bounded exploration in a continuous latent space.<n>LaTER projects final-layer hidden states back to the input embedding space, preserves the latent KV cache, and uses entropy and model-native stop-token probes to decide when to switch.
  arXiv  Detail & Related papers  (2026-05-08T06:23:58Z)
• Towards Efficient Large Language Reasoning Models via Extreme-Ratio Chain-of-Thought Compression [55.63153956934198]
  Chain-of-Thought (CoT) reasoning successfully enhances the reasoning capabilities of Large Language Models (LLMs)<n>Existing CoT compression methods often suffer from a critical loss of logical fidelity at high compression ratios.<n>We propose a novel EXTreme-RAtio Chain-of-Thought Compression framework, termed Extra-CoT, which aggressively reduces the token budget while preserving answer accuracy.
  arXiv  Detail & Related papers  (2026-02-09T06:57:15Z)
• SyncThink: A Training-Free Strategy to Align Inference Termination with Reasoning Saturation [11.021989271617835]
  We present SyncThink, a training-free and plug-and-play decoding method that reduces Chain-of-Thought overhead without modifying model weights.<n>We find that answer tokens attend weakly to early reasoning and instead focus on the special token "/think", indicating an information bottleneck.<n>Experiments on GSM8K, MMLU, GPQA, and BBH across three DeepSeek-R1 distilled models show that SyncThink achieves 62.00 percent average Top-1 accuracy.
  arXiv  Detail & Related papers  (2026-01-07T07:00:15Z)
• RaCoT: Plug-and-Play Contrastive Example Generation Mechanism for Enhanced LLM Reasoning Reliability [12.67288560758937]
  We propose RaCoT (Retrieval-aware Contrastive-of-Thought), a novel framework that shifts contrastive thinking to the pre-retrieval stage.<n>RaCoT guides the model to proactively focus on the critical details that determine answer divergence"
  arXiv  Detail & Related papers  (2025-10-26T15:06:44Z)
• VeriThinker: Learning to Verify Makes Reasoning Model Efficient [52.74493506816969]
  Large Reasoning Models excel at complex tasks using Chain-of-Thought (CoT) reasoning.<n>Their tendency to overthinking leads to unnecessarily lengthy reasoning chains.<n>We introduce VeriThinker, a novel approach for CoT compression.
  arXiv  Detail & Related papers  (2025-05-23T14:17:56Z)
• Not All Tokens Are What You Need In Thinking [34.767739567093656]
  Conditional Token Selection (CTS) identifies and preserves only the most essential tokens in chains of thought.<n>CTS effectively compresses long CoT while maintaining strong reasoning performance.<n>Further reducing training tokens by 42% incurs only a marginal 5% accuracy drop while yielding a 75.8% reduction in reasoning tokens.
  arXiv  Detail & Related papers  (2025-05-23T12:41:29Z)
• R1-Compress: Long Chain-of-Thought Compression via Chunk Compression and Search [61.4807238517108]
  Chain-of-Thought (CoT) reasoning enhances large language models (LLMs) by enabling step-by-step problem-solving.<n>CoT's extension to Long-CoT introduces substantial computational overhead due to increased token length.<n>We propose R1-Compress, a two-stage chunk-level compression framework that preserves both local information and coherence.
  arXiv  Detail & Related papers  (2025-05-22T16:06:59Z)
• Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chains [24.805434364781306]
  We introduce Compressed Latent Reasoning (CoLaR), a novel framework that dynamically compresses reasoning processes in latent space.<n>CoLaR achieves 14.1% higher accuracy than latent-based baseline methods at comparable compression ratios.<n>Our RL-enhanced CoLaR demonstrates performance gains of up to 5.4% while dramatically reducing latent reasoning chain length by 82.8%.
  arXiv  Detail & Related papers  (2025-05-22T11:40:26Z)
• Fractured Chain-of-Thought Reasoning [61.647243580650446]
  We introduce Fractured Sampling, a unified inference-time strategy that interpolates between full CoT and solution-only sampling.<n>We show that Fractured Sampling consistently achieves superior accuracy-cost trade-offs, yielding steep log-linear scaling gains in Pass@k versus token budget.
  arXiv  Detail & Related papers  (2025-05-19T11:30:41Z)
• TokenSkip: Controllable Chain-of-Thought Compression in LLMs [30.241525217939298]
  Chain-of-Thought (CoT) has been proven effective in enhancing the reasoning capabilities of large language models (LLMs)<n>We propose TokenSkip, a simple yet effective approach that enables LLMs to selectively skip less important tokens, allowing for controllable CoT compression.
  arXiv  Detail & Related papers  (2025-02-17T17:37:26Z)
• CoT-Valve: Length-Compressible Chain-of-Thought Tuning [50.196317781229496]
  We introduce a new tuning and inference strategy named CoT-Valve, designed to allow models to generate reasoning chains of varying lengths.<n>We show that CoT-Valve successfully enables controllability and compressibility of the chain and shows better performance than the prompt-based control.
  arXiv  Detail & Related papers  (2025-02-13T18:52:36Z)
• Vision-centric Token Compression in Large Language Model [51.92055188780033]
  Vision Centric Token Compression (Vist) is a slow-fast compression framework that mirrors human reading.<n>On eleven in-context learning benchmarks, Vist achieves the same accuracy with 2.3 times fewer tokens, cutting FLOPs by 16% and memory by 50%.
  arXiv  Detail & Related papers  (2025-02-02T13:10:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.